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ABSTRACT 


During  1980-82,  five  studies  of  holistic  writing  assessment  pro- 
cedures were  conducted  in  several ^school  districts  in  northern  California • 
These  studies  sponsored  by  the  Bay  Area  Writing  Project  under  a  grant  from 
the  National  Institute  of  Education  examine  interactive  relationships  of 
the  participants,  processes,  and  products  of  writing  assessment  episodes. 
The  first  study  examines  practices  in  designing  writing  test  prompts  and 
presents  a  report  of  these  practices  that  was  reviewed  by  a  panel  of 
teachers,  researchers,  and  professional  test  makers.    This  study  also 
reviews  the  literature  from  major  fields  related  to  the  testing  of  writing. 
The  second  study  investigates  the  effects  of  variation  in  the  specifica- 
tion of  audience  in  a  writing  test  prompt  upon  holistic  score.    A  second 
part  of  this  study  examines  rater  response  to  the  anchor  papers  used  in 
training  three  groups  of  raters  for  scoring.    The  third  study  investigates 
writing  task  and  response  variables  which  potentially  result  in  discrep- 
ancies in  holistic  scores,  and  validates  an  analysis  system  for  studying 
the  relationships  between  assessment  prompts,  written  products,  and  holis- 
tic scores.    The  second  part  of  the  third  study  summarizes  data  collected 
on  the  performance  elicited  on  eight  prompts  in  a  county-wide  writing 
assessment.    The  fourth  study  investigates  differences  in  expectations 
among  participants  in  a  writing  test  episode,  and  includes  an  analysis 
of  characteristics  of  a  prompt  which  influence  variations  in  interpreta- 
tion.    The  fifth  study  investigates  holistic  assessmei1l|b  as  a  longitudinal 
measure  of  student  growth,  focussing  on  aspects  of  performance  variation 
over  time.    A  discourse  typology  is  developed  and  applied  in  an  analysis 
of  performance  variation  on  annual  assessment  fopics  over  a  period  of  five 
years. 
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PROPERTIES  OF  WRITING  TASKS: 

A  STUDY  OF  ALTERNATIVE  PROCEDURES  FOR 
HOLISTIC  WRITING  ASSESSMENT 

EXECUTIVE  SUMMARY 
Leo  Ruth»  Project  Director 


Background  and  Rationale 

Evaluation  Is  an  essential  step  In  the  process  of  In- 
struction and  In  determining  the  accomplishments  of  school 
programs.     The  evaluation  model  being  refined  In  the  Bay  Area 
Writing  Project's  study  of  writing  assessment  Is  the  so-called 
"holistic"  approach  to  appraisal  of  t^e  writing  samples 
produced  In  response  to  given  prompts.     Holistic  assessments 
test  writing  by  calling  for  a  student  to  produce  an  actual 
sample  of  writings  which  later  Is  read  and  rated  according  to 
systematic  procedures  variously  derived.     The  rapid  growth  of 
this  style  of  writing  assessment  has»   however,  outstripped 
knowledge  of  how  components  of  the  model  Interact.  Needed 
Is  a  comprehensive  study  of  Interactions  between  writing  task, 
student  response,   and  reader's  rating  In  the  course  of  making 
a  holistic  writing  assessment. 

Often  the  first  question  asked  when  evaluators  embark  on 
a  holistic  assessment  is  "What  topic  should  we  use  and  how 
should  we  word  it?"     The  writing  task,  whatever  its  nature, 
serves  as  the  prompt  or  the  stimulus  for  the  writ Ing, perfor- 
mance evoked.     Inasmuch  as  the  prompt  stands  in  dynamic  rela- 
tion to  the  piece  of  writing  it  Initiates,  evaluators  pre- 
sumably would  desire  to  know  how  particular  properties  of 
prompts  might  affect  the  performance  of  writers  in  both 
Intended  and  unintended  ways.     Tet  surveys  of  relevant  litera- 
ture reveal  surprisingly  little  theory  or  data  regarding  the 
properties  of  writing  prompts. 

Given  the  long  history  of,  the  use  of  written  examlna t ions 
in  this  country,  one  might  expect  to  find  a  body  of  knowledge 
drawn  from  theory,  research,  and  practice  which  guides  the 
design  of  the  prompts   (topics,  tasks,  assignments,  exercises, 
questions,   instructions,  visual  stimuli)  used  to  elicit 
specimens  of  writing  for  purposes  of  evaluation.     Yet,  the 
search  of  the  literature  presented  in  Chapter  1  reveals  that 
relatively  few  details  are  yet  available  from  educational  or 
psychometric  research  against  which  one  might  check  current 
practice  in  designing  writing  assessment  prompts.  Various 
researchers,  in  the  field  of  writing  have  noted  this  particular 
gap  in  the  available  research  on  writing  assessment  (Odell, 
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Cooper,  and  Courts.  1978;  Odell,  1979j  Greenberg,  1981; 
Hoetker,  1981). 


Psychometr Ics ,   for  example,   has  dealt  with  a  restricted 
range  of  concerns  in  relation  to  developing  vr It Ing  promp t s • 
Textbooks  on  measurement  commonly  deal  with  test  construct  ion » 
but  their  focus  tends  to  be  more  on  the  technology  and  metho* 
dology  of  measurement  than  on  the  analysis  of  substantive 
properties  of  the  essay  questions  that  comprise  writing  tests. 
Psychome tries  provides  methods  of  applying  statistical 
approaches  to  the  appraisal  of  the  "behavior"  of  writing 
prompts  or  objective   test  Items  during  field  trials  and  actual 
test  administrations.     But  psychome  tries  does  not  Ojffer 
guidance  beyond  the  most  general  sorts  of  principles  for 
ac::ually  designing  and  wording  the  writing  tasks. 

Through  the  agency  of  a  "field  trial,"  psychometr Ics 
provides  a  self-evaluative  procedure  for  eliminating  faulty 
essay  questions  or  other  test  items,  but  psychome tries  does 
not  have  empirically  derived  analytic  methods  for  determing 
what  has  gone  wrong  in  framing  the  essay  question   (Payne  and 
McMorris,   1967).     The  essay  questions  that  survive  the  field 
trial  are  presumed  to  be  "meaningful"  to  the  larger  population 
to  be  tested.     Though  psychometr ics  strives  to  be  scientific, 
the  procedures  used  to  construct  and  validate  essay  questions 
are  essentially  subjective  and  scientifically  crude  in  the 
sense  that  there  is  no  theory  of  information  processing  or  of 
cognition  underlying  the  design  of  writing  tasks. 

Standardized  testing  procedures  pre-suppose  that  the 
examiner  and  the  student  writer  share  a  common  language  and 
uniform  interpretations  of  tasks.     Yet,  Mellon,   for  example, 
in  his  discussion  of   the  first  National  Assessment  of  Educa- 
tional Progress  Writing  Assessment  commented  on  the  difficulties 
of  the  NAEP  assessors  in  formulating  writing  topics  ^even  with 
the  help  of  experienced  personnel  from  the  Educational  Testing 
Service).     Mellon  wondered  why  a  seemingly  straightforward 
topic  like  "Going  to  School"  produced,  on  the  part  of  some 
writers,   such  mundane  accounts  of  scenes  along  the  way,  while 
it  elicited  descriptions  of  fantasy  occurences  in  others .  This 
is  only  one  example  selected  from  many  that  clearly  indicates 
particular  topics  do  not  guarantee  the  activation  of  particular 
levels  of  thought  and  forms  of  discourse.     The  function  of  a 
writing  task  as  a  s t imulus  event  cannot  be  understood  until  the 
examiner  determines  how  constraints  of  language  use  and 
interpretation  are  operating  in  the  writing  assessment  situation 

Despite  the  fact  that  researchers  in  educaticaal  measure- 
ment have  acknowledged  question-wording  effects   (French,  1966; 
Coffman,   1971),   the   larger  theoretical  issues  of  essay  task 
wording  and  typology  have  seldom  been  addressed.     There  has 
been  little  research  to  determine  the  frequency,  magnitude,  or 
underlying  nature  of  essay  task-wording  effects.  Wording 
effects  have  tended  to  be  treated  anecdotally  or  incidentally 
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in  the  context  of   other  objectives  of  writing  measurement  study. 
(For  example,   see   anecdotal   treatments  of  wording  effects  in 
Godshalk,   Swineford,   and  Coffman,   1966;  Winters,   1980;  Smith, 
1980;   Lloyd-Jones,    1977;   Michigan  Survey  Research  Center 
Interviewer  Manual ,   1969  ,   etc.)     Current  procedures  for  "item 
analysis"   do  not   lead  to  "analysis"   of   the   linguistic  Droper- 
ties   inside   the  texts  of   essay  questions  which  might  affect 
their  interpretations  by   the  persons  responding  to  them.  Sax 
acknowledges   that    the  procedures   called  "item  analysis"  are 
such  that  "unfortunately   [they]    cannot  reveal   if  ambiguity 
results   from  the   students'    lack  of  knowledge  or   from  poorly 
written  items"   (1974,  p.    233).  , 

There  are,  however,    two  other  sources   that  promise  useful 
information   to  guide   the   development   of  writing  prompts. 
These  sourj;es  are   discourse  theory  and  practitioner  knowledge. 
Discourse   theory,   which  deals  with  the  development  of  full 
texts   in  speech  and  writing   (Kinneavy's  definition,  1971)* 
does  provide  a  source  of   knowledge  that  influences  the  design 
of  writing  assignments  for  both   instructional  and  testing  — 
occasions.     Discourse  theory,  with  roots  reaching  back  to 
antiquity,   attempts  to  describe   the  varieties  of  discourse 
and  the  elements  which  contribute  to   the  making  of  texts, 
spoken  and  written.     Various  schools  of  thought  have  emphasized 
one  or  another  element  such  as   the  wr i ter ,    the   subject ,    or  the 
or   the  audience,   either   individually  or  taken  together.  A 
fourth  element,    f  unction    (also   called  purpose  or  aim)  arises 
from  the  interaction  of  writer,   sub j  ec t ,   and  audience .  Related 
to   all  of   these   is   the  concept   of   form   (also  called  mode  or 
genre) .     The  basic   problem  ind  dealing  with  this  field   is  the 
very  richness  of   the  area  and  the  problems  attendant  upon 
selecting  from  among  competing   theories.     These  various  dis- 
course theories  provide  the  categories  of   forms  and  functions 
of  writing  that  frequently   turn  up  as  specifications  and 
constraints  in  writing  prompts. 

Practitioner  knowledge  tends  to  be  undervalued:     It  has 
neither  the  status  of   theory  nor  of  empirically  tested  know- 
ledge.    Yet  practitioner  knowledge  develops  through  a  con- 
tinuing involvement   in  the  unique  and  particular  experiences 
of   the  profession.     This  practical  knowledge  may  denote  a 
high  level  of   competence,    but   it  remains  essentially  an  indi- 
vidual theory  of   practice   so  long  as   it  remains  private.  How- 
ever,  some  of   the   privately-held  theories  of  practice  eventually 
"go  public"   through  textbooks,    curricula,   and  other  media.  Once 
public,   these  works  are  subject   to  critical  processes  which 
may  lead  to   their   verification,   validation,   and  refinement. 
Thus,   in  time,   there  develops  a  form  of   theory  that  might  be 
called  "a  theory  of   idealized  practice."     This  source  offers 
knowledge   to  use   in  developing  normative  "rules"   that  may  be 
recommended  on  the  basis  of  successful  practice  for  designing 
pr  omp  ts.     The  practical  wisdom  available  from  the  exper  ience 
of   professionals   in   topic  writing  needs  to  be   collected,  sorted, 
appraised,   and  used   to  formulate  propositions   to  guide  the 


ERIC 


2 


desi^gn  of  writing  test  prompts. 
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There  are  Indications  in  the  literature  that  researchers, 
evaluators,   and  teachers  are  not  sufficiently  aware  of  the 
nature   and  range  of   task  effect  upon  writer  performance. 
Bereiter,   Scar damalia ,   and  Bracewell  have  noted: 

Developmental  re  search  in  writing  has  generally 
proceeded  by  assuming  that   the  task  was  cons  trued 
the  same  by  all  subjects  and  therefrom  inferring 
differences  in  competence*     But   this  is  quite  an 
inadequate  way  of  going  after  any  deep  understanding 
of  cognitive  development  in  writing   (1979,   p*  5, 
as  quoted  in  Polin,   1980,   p.  8). 

Local  school  assessors  of  ten  provide  an  all-purpose  essay 
topic  and  generalize  broadly  from  the  student's  response  to 
this  single  measure  to  predict  performance  on  any  writing 
occasion,   assuming  that  "good  writing"  is  a  unitary  construct, 
not  subject  to  differences  in  topic  and  purpose.  Similarly, 
some  schools  offer  topics  intended  to  stimulate  exposition 
or  argument  but  which  instead  measure  thit  writer's  general 
knowledge  rather  than  his/her  capacity  to  mount  an  argument 
or  sustain  a  piece  of  exposition. 

In  addition,   many   testers   insist   that  "audience  should  be 
specified,   "but   they   fail  to  make  distinctions  about  what  kinds 
of  audience*     A  California  writing  assessment  once  specified 
audience  as  follows:     "Write  a  letter  to  your  pen-pal."     But  a 
study  by  Marion  Crowhurst   (RTE,   1978)   found   that  students  pro- 
duce more  sophisticated  and  complex  syntax  when  writing  for  a 
teacher  or  general  reader  than  when  writing  for  a  peer.  If 
testers  want   to  evaluate  maturity  of  syntax,   then  a  prompt 
which  specifies  an  unsophisticated  audience  is  counterproductive. 

The  writing  stimulus,  whatever  its  nature,   is  a  spring- 
board.    Researchers  and  assessors  must  not   lose  sight  of  the 
linguistic,   cognitive,   and  social  reverberations  it  may  set  off 
in  the  writer.     Yet,   as  far  as  we  can  determine,   there  are  a 
number  of  interactions  between  writing  stimulus  and  writers 
that  have  not  yet  been  explored.     At  issue  is  the  nature  of  the 
interaction  between  the  topic  and  any  accompanying  instructions 
(written  or  verbal)   and  what  these  may  "prompt"  a  writer  to 
produce.     Since   it  is  known  that  choice  of   topic  with  instruc- 
tions can  affect  the  performance  of  the  writer,  how  do  we  know 
whether  the  prompt  given  is  actually  causing  the  student  to 
perform  either  typically  or  as  well  as  he  actually  is  able  to 
perform     in  the  areas  we  would  like  to  assess? 

There  are,   then,   potential  sources  of  invalidity  in  the 
conduct  of  holistic  assessment  at  the  point  of  interpreting 
the  writing  task  and   in  the  reading  of   the  writing  samples 
produced.     There  is  a  potential  for  possible  miscommunicat ion 
in  the  writer's  interpretation  of  and  response  to   the  writing 


problem  set   in  a  holistic  writing  assessment.     These  are  the 
issues   that  have  been  addressed  in  a  series  of  inter-related 
research  studies  presented   in   the   following  report.  The 
several  studies  provide  data,    theory,   and   recommendations  for" 
practice   that  will  enable   the   authors  of  writing  assessment 
prompts  to  understand  more  exactly   the  nature  of   cues  and 
constraints  within  the  prompts  and  how  these    elements  affect 
the  performance  of   students  writing  the  papers  and  the 
Judgments  of  evaluators  reading  the  papers.     The  data  and 
recommendations   from  these   studies   are  expected   to  have 
significant   implications   for  practices   in  the  generating  of 
prompts   intended   to   initiate   the  display  of   competence  in 
writing  assessments. 


The   Research  Problem 


The  principal  research  purpose  governing   the  work  of  all 
dimensions  of   this  NIE/BAWP  Writing  Assessment  Project   is  to 
increase  our  knowledge  of   the  properties  of  writing  test 
prompts   and   the   compositions  they  Initiate   in  holistic  writing 
assessments.     There   is  considerable  evidence   to  enable  us  to^- 
"  conclude  that  particular  prompts  do  not  perform  as  uniform 
stimuli  across  populations  on  various  testing  occasions  to 
activate  expected   levels  of   thought  and   forms  of  discourse. 
This   investigation  asks   the  questions: 

1.  What   elements   in  the   content  and  structure 
of  writing   test  prompts   influence  the 
production  of   compositions   during  writing 
assessments? 

2.  What  makes   a  prompt   "good"   for  eliciting 
writing   samples   to  be  rated  under  conditions 
of   "holistic  scoring"  procedures? 

3.  How  are  "good"  writing  prompts  generated  and 
tested  systemat  ically ? 

A.     What   changefs  occur   in  maturing  students' 

responses   to  writing  prompts   in  successive 
annual   assessments,   and  what  do   these  changes 
reveal  about  how  ve  should  be  assessing 
developing  abilities   in  writing? 

5.     What   sources  of  knowledge  do  we  have   to  guide 
current  practice   in  designing  effective 
writing  prompts? 

The  several   chapters  of   this  report  address   these  basic 
research  questions. 


Description  of   the  Research 


Theotetlcal  Grounding  of   the  Research^     Writing  research 
has  only  recently  emerged  as  a  distinct  field  of  Inquiry.  As 
an  Interdisciplinary  field   It  draws  from  branches  of  linguistics, 
psychology,   anthropology,   education,   cognitive  science,  social 
science,   and  literary  study.     It  would  be  difficult  to  trace  In 
this  summary  of  our  report  all  the  crossings  of  discipline 
boundaries  undertaken  within  each  of   the  five   studies  presented 
here.     But   the  particular   significance  of  One   theoretical  field 
Is  clear:     we  noted   In  our  research  proposal   that  the  act  of 
writing  begins  with  an  act  of  reading  comprehension  as  the 
writer  encounters  the  text  of  a  prompt .     Given  our  observation 
of   the  variability  of  Interpretations  of  writ  In g  tasks,  we  were 
drawn  to   the  construe tlvlst   theories  of  reading  which  we  began 
to  adapt   Into  an  evolving  construe tlvlst  model  of  writing  task 
compr ehens Ion . 

Our  conception  of  the  nature  of  reading  comprehension  has 
been  changing  within   the  last  few  years  as  a  result  of  theoreti- 
cal and  experimental  work  In  fields  such  as  psychollngulstics, 
cognitive  psychology,   artificial  Intelligence  and  other  disci- 
plines.    An  Increasing  number  of   studies  In   these  fields  have 
given  attention  to  comprehension  and  memory,   both  considered 
as  cons.tr uctlve  processes.     The  "construe tlvlst"  position 
considers  comprehension  to  be  a  complex  process  which  depends 
on  world  knowledge,   and  Intra-sen tence  content,   as  well  as 
Inter-sentence  context.     The  constructive  process  feeds  on  the 
Interaction  of  Information  presented  through  text  structures, 
the   context  of   that   Information,   and  the  existing  knowledge 
framework  of   the  reader.     Reading,   In  effect.   Is  a  specific 
application  of   general  cognitive  processing  skills  and 
strategies . 

Our  study  of  properties  of  prompts  Is  based  upoiv  analyses 
which  propose  a  con  struct Ivlst  theory  of  writing  task  Inter- 
pretation.    Writers  choose  rather  freely  ni^ong  cues  embedded 
In  the  prompt,   both  honoring  and  Ignoring  elements  which  may 
enable  the  writer,  with  varying  degrees  of  success,   to  match 
the   test-maker*s  Intention  and  expectation.     The  meaning 
potential  of  a  given  prompt   Is  thus  relative   to  the  linguistic, 
cognitive,   and  social  revdrberat Ions  set  off   In  the  writer. 
This  theory  draws  on  schema-based  theories  of  Information 
processing  and  socially-based  theories  of  communicative 
competence.     This  theory  of   the  nature  of   the  prompt  shows 
how  particular  properties  of   toplcs--ldeatlonal.  Interpersonal, 
and  textual  aspects^-f unction  simultaneously  In  the  assessment 
episode  In  shaping  the  written  responses  that  are  formed  to 
meet  the  perceived  requirements  of   the  topic. 

The  construct Ivlst  theory  of   task  meaning  being  developed 
here  has  significance  for  generating  prompts  for  holistic 
writing  assessments.     The  problem  Is  not  one  of  simply  forming 
"better"  prompts  but  of  understanding  how  a  host  of  properties 
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may  influence  what   a  writer  "makes"  of  any  prompt   given  to 
him  to   initiate  his  display  of  writing  competence:      the  same 
prompt  put  to   two  different  people  may  not  be  perceived  as 
the  same  prompt.      The  properties   of   tasks  and  what  theyelicit 
in  writing  performances   are   contingent  upon   the  language  of 
the  prompt,   the  world  knowledge  of   the  participating  writers, 
and   the   social   roles  of   the  participants   in   the  "speech  event" 
of  a  writing  episode. 

The  Model   of   the  Writing  Assessment  Episode.      In  thinking 
through  our   strategies   for  approaching  our  research  questions, 
we   found   it  useful^  to  keep   in  mind  the   simple  visual  model  of 
an  asses  sment   episode  as   depicted  below.     There  are  varying 
degrees   of   abstraction  and   formalization  in   the  construction 
of  models.     Some  achieve   the  strength  of  predictive  models  of 
explanations   of   scientific  phenomena.     We  created  ours,  how-^ 
ever,    as  a  scheme   for  consolidating  and  ordering  the  concepts 
we  wanted   to  keep   in  mind.     For   a  model  makes  knowledge 
accessible   for   display,  manipulation,   and   interpretation.  As 
a  manageable  representation  of  entities   in   the  real  world  that^^ 
it  represents,   a  model  becomes  a  useful  guide  for  making 
interpretations  and  decisions  about  the  real  world   it  repre- 
sents.     In  this   case,  we  wanted   to  keep   in  mind   the  unity  of 
the  assessment   event  and   the  interactive  relationships  of  the 
participants,   processes,   and  products   that  comprise   it.  So, 
early   in  the  project  we  began  with  a  much  simplifie^^  version 
of   the  model  now  presented  here.     This  version  is  still 
labeled   "tenative,"   indicating  that  it   invites  further 
modification.      Essentially,    there  are   three  basic  processes 
depicted   in  our  model  that  we  sought   to  understand    (see  following 
page) . 

1.  THE  PROCESS  OF  DESIGNING  THE  PROMPT.     What  goes 
on  in   the   test -maker 's  mind  during   the   const ruc- 
tion  of   a  prompt?     What  wording  choices   does*  the 
tester  make  toconvey  accurately  his/her  inten- 
tions?    How  does  the  writer  of   the  prompt  make 
decisions   relating   to  rhetorical  specifications 
of  mode,   role,   audience,   purpose?     What  assump- 
tion  about  world  knowledge  and  cognitive 
developmental   level  of   the  writer   does  the 

test -maker  hold  ? 

2.  THE  PROCESS  OF  PERFORMING/COMPOSING.     What  does 
the  student  writer  understand  his/her   task  to  be? 
Which   of   the  tester *6  cues  and  constraints  does 
the  student  writer   recognize,   understand,   and  use? 
Given   the  writer *6   interpretation  of   the  task, 
how  does  he/she  go  about  accomplishing  it? 

3.  THE  PROCESS  OF  RATING.      What   does   the  evaluator 
understand   the   intention  of   the  prompt   to  be? 
How  does   the  evaluator   interpret   the  intention 
of   the  writer   in  relation  to   the  evaluator *s  own 
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AN  EVOLVIHG  HOOEL  FOR  STUDYING  THE  KRITIKG  ASSESSMENT  EPISODE: 
}>ARTI  CI  PA«TS^^mESsis?^^?'^C(:XTS^ 
 1 


IttiE  TEST-MAKE?r|-T^ 
Role: 


Composer 
of  Text 


Underlying  Factors 

Assessment  purpose 
Assunptlons 
Expectations 
^Intended  weaning  i 
language  knowledge 
Rhetorical  model  (s) 
AssuRied  Morld 
knowledge  etc. 


^^MtexToi  . 

ji^RITING  PROMPTI 

'  Features 
Explicit 


Topic/Subject 

Specifications 

Tone 

Structure 
Information 
load 

Languagt 

Implicit 


Inference 
Unintended 
effects 
Ambiguity 
Mixed  messages 
Etc. 


THE  TEST-TAICER 


RoTes: 

Comprehender  of 
llrlting  Prompt 

Comprehender  of 
Writing  Specimen] 

Underlying  Factors 


Testing  context 

Assunptlons 

Expectations 

DTO£rpoo?jMnTs3 
Perceived  task 
Control  of  code 
Processing  state 
Inner  motivation 
Actuel  world 
knowledge  etc. 


I 


TEXT  OFr** 

laftniNG  sPEcTBCTi 


Features 

Explicit 

Topic/Subject 
Voice 

Gender  chclct 
Structure 
Information 
Language 
control 

Implicit 

Unintended 
effects 

Hlxed  messages 
about  audience, 
tone 

Etc. 


P"|n<E  test^ater] 
Role: 

Comprehender  of 
Mrlting  Prompt 

Comprehender  of 
Mfrlting  Specimen 

Evaluator  of 
Urltlnj  Specimen 

Underlyint  Factors 


Understood  function! 
Assumptions 
Expectations 

Understood  meanliig 
ercelved  task 
Rhetorical  model 
^orld  knowledge 
91is 

Notion!  of  correctness 

 J 

f 

t 
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  •  Cognitive  Processes 

 •  Visible  Product 

•  The  besic  categories  cf  the  model  were  proposed  by  the  project  director,  Leo  Ruth,  at  a  staff  meeting  on  10/22, 
1980.  K«ny  refinements  of  the  current  version  reflect  the  contributions  of  the  participants  In  our  weekly  con- 
C6?tu^l  brainstorming  meetings,  attended  In  the  first  year  of  the  project  (1980-81)  by  Sandra  Murphy,  Catherine 
Keech,  Paul  ^nnon•  Judith  Langer,  Donald  Leu,  Charles  Kinxer,  and  Karen  Carroll,  and  other  visitors. 


perception  of  the  task?     How  do  the  evaluators' 
biases- -preferred  discourse  model,   notion  of 
the  relevance  or  ^irrelevance  of  lower  order  ' 
scribal  skills,  preferences   for  displays  of 
creativity,   originality,   reasoning  or  the 
lack  of  these,   opinions  and  knowledge  of ^ the 
world  —  enter   into  phe  ^.assigning  of  quality 
ratings? 

The  Principal  Studies 

Th(B  principal  studies  of   the  NIE   funded  Bay  Area  Writing 
Proje^c^  Writing  Assessment  Project  are  listed  below: 

1.  Designing  prompts  for  holistic  writing  assessments: 
Knowledge  from  theory,   research  and  practice,  by 
Leo  Ruth  and  Catharine  Keech 

2.  Effects  of  variation  in  a  writ-ing  test  prompt  upon 
holistic  score  and  other  factors,  by  Catharine 
Keech,  Don  Leu,   Sandra  Murphy,,  and  Charles  Kinzer 

3.  Beyond  the  field  test  of  writing  prompts:  a 
preliminary  investigation  of   the  nature  of  response 
variation  to  selected  field-tested  ptompts,  by 
Charles  Kinzer  and  Sandra  Murphy 

4.  A  study  of  the  construction  of   the  meanings  of  a 
writing  prompt  by  its  authors,    the  student  writers, 
and  the  raters  by  Sandra  Murphy,   Karen  Carroll, 
Charles  Kirizef,   and  Ann  Robyns 

5*     Holistic  assessment  as  longitudinal  measure  of 
student   growth:     Interpreting  stTudent  task 
constructions,   by  Catharine  Keech  and  Elissa  Warantz 
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HIGHLIGHTS  OF  THE  STUDIES 
PROPERTIES  OF  WRITING  TASKS   IN  HOLISTIC  WRITING  ASSESSMENTS 


STUDY  1.      DESIGNING   PROMPTS   FOR  HOLISTIC  WRITING  ASSESSMENTS: 
KNOWLEDGE  FROM  THEORY,   RESEARCH  AND  PRACTICE 

Part   I:      Sources  of  Knowledge  for  Designing  Writing 
Test  Prompts 

i 

Leo  Ruth 

A .  Tbe  Problem 

1.  What  .sources  of  knowledge  do  we  have   to  guide  current 
practice  in  designing  effective  writing  prompts: 

2.  What  makes  a  prompt   "good**  for  eliciting  writing  samples 
to  be  rated  under   conditions  of  "holistic  scoring?** 

3.  How  are  '*^ood'*  writing  prompts  generated  and  tested 
systematically? 

B.  Outcomes 


1.  Five  principal  sources  of  knowledge  were  studied: 

a.  Psychomet r ics 

b.  Prof essional*  wisdom 

c.  Discourse  theory 

d.  Writing  res  ear ch 

e.  Selected  Practices 

2.  The  selected  practices  were  reviewed  in  draft   form  by 
a  panel  of   teachers,   researchers,   and  professional 
tes  t-maker s . 

3.  The  report  on  practice,   revised  in  the  light  of 
crit l/q-reviewers  *   recommendations  appears  as  ^ 
Part  Vl^  of   this  study. 

4.  Ideas   from  the  other  four  principal   sources  of 
knowledge  about  writing   test  prompts  are  summarized 

^  in  four   guides    beginning  at  the  end  of  Part   I,  Chap.  1, 

a.  The  Philosophy  of  the  Writing  Assignment  for 
Assessment 

b .  Gjjiidel  ines   for  developing  Prompts  for  Writ  in  g 
Assessment 

c.  A  Tentative  Schema  of  'Writing  Tasks  for  the 
Development  of  Assessment  Prompts 

d.  Schema   for  Task  Analysis  of  Prompts 

C .      Issues   and  Recommendations 

1.     The  manipulation  of  variables   is  problematic  in 
^.-.experimental   research.     The  notion  of  the  introduction 

of  a  *'variable**  into  a  prompt  presupposes  that  it  is 

perceived  as   something  significant. 
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There  is  currently  no  well-established  rational, 
rule-governed  procedure  for  generation  of  writing 
test  prompts.     We  need  to  discover  and  test  systematic 
and  theoretically  sound  rules  which  enable  us  to 
design  appropriate  prompts  for  given  tes.t-wr i ter- 
task  occasions.     But  we  need  to  be  wary  of  going  so 
far  that  we  t.yrn  the  prompt  development  process  into 
a  "paint-by-numbers"  approach. 

Tliere  &re  a  number  of   text  characteristics  that  might 
be  taken  into  account: 

a.  Text  length 

b.  Density  of  information 

c.  Density  of  nfew  information 

d.  Density  of  arguments  in  propositions 

e.  Capacity  to   inspire  interest 

f.  Cohesive  features 

g.  Presence  or  absence  of  information 

Prompts  need  also   to  be  examined  for  the  cognitive 
demands  they  make: 

a.  Language  production  requirements 

b.  Presupposed  genre  knowledge 

c.  Memory  and  retrieval  requirement s 
d^.   Reasoning,  requirements 

e.  Explicitness  of  task  demands  and  potential  for 
recognition 

f.  Developmental  factors  in  interpretation 

g.  Motivational  potential 

h.  Presupposed  world  knowledge 

•  * 

Field  trials  are  essential,  but  new.steps  need  to  be 
taken.     Instead  of  merely  d^eyeloping  statistical 
indexes  of  difficulty,  we  need  to  go  beyond^the  trial 
to   conduct   task  analyses  and  find  Out  why  writfers 
responded  the  way  they  did. 

Raters  and  evaluators  need  to  recognize  a  range  of 
allowable  variation  in  the  writer's  interpretation 
and  response  to   the  prompt. 

Implications  for  teaching  include: 

a.  Teaching  students  how  to  "read"  a-  writing  test  prompt 
r-how  to  separate  out  task  co.mponents 

--how  to  read  for  intention,  implication 

b.  Teaching  students  strategies  for  managing  constraints 
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Understanding  the  distinctions  between  "real" 
writing,   "school"  writing,   and  "test"  writing 
Understanding  that  the  process  of  writing  under 
time   constraints  in  a  test   situation  is  different 
from  natural  processes  of  wr  iting . 
Teaching  students  how  to  achieve   flexibility  in 
the  range  of  roles   they  can  assume  and  audiences 
they  can  address. 
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Part  II:     Practices  in  Designing  Writing  Test 

Prompts :     Analysis  and  Recommendations 


Catharine  Keech 


The  Problem 

1.  How  should  students  be  asked  to  write  and  what  should 
they  be  asked  to  write  about  if  we  wish  to  measure 
their  real  writing  abilities? 

2.  What  basic  principles  guide  professional  test -makers 
in  ai^swering  the  first  question? 

Outcomes 

1.     The  entire  report  constitutes  the  outcome  of  the  study 
What  follow  are  only  a  few  of  the  interesting  findings 

Selection  of   the  Subject   (and  Problems  of  Prior 
Knowledge) 

a.  Topics  that  encourage  introspective  or  autobio- 
graphical writing  by  drawing  on  personal  experience 
are  both  motivational  and  intentional* 

b.  **Choosing  interesting  topics  and  expressing  them 
simply  will  ensure  that  most  students  can  find 
something  meaningful  to  say  in  their  writing.** 

c.  Content  specifications  can  "cue"  students  to  see 
the  full  potential  of  the  topic,   explain  the  writer 
options,   and/or  clarify  the  tester's  exp.ec  tat  ions  * 
However,   there  are  two  dangers  of  elaborate  cuing: 

1.  Poor  readers  may  be  put  off  by  the  sheer  amount 
of  text;  and 

2.  Students  could  mistake  the  cuing  as  "commands" 
rather  than  as  "suggestions  •" 

d.  Hypothetical  situations   (asking  students  to  Imagine 
pretend,   or  wish)  make  good  subjects  for  writing; 
however ,   the  need  to  use  the  conditional  or  sub* 
Junctive  verb  phrases  can  present  problems  for 
some  students • 

e.  Problems  relating  to  differences  In  prior  knowledge 
among  students  can  be  avoided  by  providing  a  body 
of  relevant    texts  or  a  ilterary  passage  to  be  com- 
mented on  or  argued  from* 
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f .    Controversial  Issues  can  provide  especially 

appropriate  topics  for  tests  of  persuasive  writing. 
But  "controversial  topics  can  create  special 
problems  for  holistic  assessor s  ^f   the  subject 
arouses  strong  biases  In  both  student-writers  and 
teacher  ^readers'*  because  readers  can  be  distracted 
from  the  task  of  evaluating  writing  ability  by 
the  nature  of  the  written  content. 


Issues  and  Recommendations 

The  report  is  filled  with  many  recommendations  and  discusses 
many  Issues.     Only  a  small  sample  of  these  is  provided  here. 

1.  The  "describe  trap"  can  be  avoided  by  substituting 
the  instruction,  "Tell  about." 

2.  Instructions  are  useful,  but  too  many — i.e.,  overprompt- 
ing--*can  become  constraints. 

3.  A  very  helpful  constr-aint  in  instructions  is  a  suggestion 
to   focus  on  one  aspect  of  a  larger  topic. 

4.  The  "special  conventions  trap"  can  result  in  student 
misunderstanding  of  the  intent  of  the  assessment — 
i.e.,  is  knowledge  of  the  particular  convention 
being  tested  as  well  as  the  overall  ability  to  write — 
and  too  much  attention  gets  paid  to  the  formal  aspects 
(e.g.,   the  business  letter)  rather  than  to  the  production 
of  a  good  piece  of  writing. 

5.  The  "pick-a-form  trap"  may  seem  free,  but  teacher- 
raters  often  have  preferred  forms  that  get  rewarded, 
while  other  forms  are  penalized* 

6.  Specifying  number  of  works,  sentences,  paragr&phs 

is  an  unnecessary--and  usually  misunderstood — constraint 
that  does  not  qualify  as  helpful  instructions.  Such 
specification  inhibits  fluency,  and  emphasizes  minimum 
rather  than  optimal  performance. 

7.  Specifying  audience  may  be  useful  in  some  testing 
situations,  but  can  cause  problems  in  others.  Merely 
mentioning  an  audience  is  not  sufficient.  "When 
audience  is  specified  in  a  writing  task  that  provides 
a   full  rhetorical  context  and  a  clear  purpose  for 
writing,    the  effect  is  to  produce  a  more  realistic 
writing  problem. " 

8.  Avoid  hidden  biases  and  loaded  wor^    and  phrases. 
Students  sensing^  the  bias  may  wrinte  from  their  conception 
of  what   the  te'st-mak^r  believes,   rather  than  from  their 
own  perceptions  and  opinions. 

3,:  ' ' 
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STUDY  2.  EFFECTS  OF  VARIATION  IN  A  WRITING  TEST  PROMPT 

UPON  HOLISTIC  SCORE  AND  OTHER  FACTORS 

Part  I:     Effects  of  Two  Versions  of  a  Writing 

Prompt  Upon  Holistic  Score  and  Writing 
Processes 


Don  Leu  t  Ca thar.ine  Keecht  Sandra  Murphy » 
and  Charles  Kinzer 

A.  The  Problem 

1.  The  purpose  of  this  study  was  *'to  investigate 
effects  of  variation  in  the  specification  oi 
audience  on  a  timed  writing  test*..*'     The  study 
involved  114  high  school  students*  grades  10-12 » 
from  one  teacher's  four  classtes. 

2.  The  day  after  students  wrote  in  response  to  one 
of  the  two  versions  of  the  prompt*   they  completed 
a  questionnaire  intended  "to  elicit  information 
about  their  attitudes  toward  the  prompt*  their 
recollections  of  the  writing  process  during  the 
timed  writing  tests*  and  their  descriptions  of 
their  other  writing  behaviors  and  attitudes* 
both  in  and  out  of  school." 

3.  Two  questions  were  of  particular  interest  in 
examining  the  data  drawn  from  the  writing  test 
and  the  questionnaire: 

a)  Which  variables  are  associated  with  high 
holistic  scores? 

b)  Which  variables  are  associated  with  differences 
in  the  prompt? 

B.  Outcomes 

Differences  in  the  two  versions  of  the  prompt  did 
not  produce  a  difference  in  the  performance*  as 
measured  by  total  holistic  score  means.     But  does 
this  mean  that  providing  additional  information 
about  the  writer's  audience  and  rhetorical  purpose 
does  not  affect  the  quality  of  performance  in  a 
tilled  writing  test? 

Students*  choice  of  writing  mode  was  affected  by 
the  different  prompt  versions.     STudenti  writing 
on  Version  B  (with  a  specified  audience)  produced 
20  alternate  mode  papers   (;L6  letters*   3  Journal 
entries*   1  essay);   students  writing  on  Version  A 
(with  an  unspecified  audience)  produced  only  seven 
alternate  mode  papers  (5  letters*  2  Journal  entries). 

3.. 
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3.  More  planning  time  was  spent  on  the  prompt  version 
with  the  less  specific  audience  definition. 

4.  Nearly  three-fourths  of  the  students  reported  that 
they  would  have  written  very  differently  if  they 
had  had  more  time.     This  suggests  that  student 
performance  on  a  shorty   timed  writing  test  may 
differ  significantly  from  performance  on  a 
regular  classroom  writing  assignment. 

Issues  and  Recommendations 

1.  "Although  one  task  constraint^  degree  of  audience 
specification^  was  manipulated  experimentally » 
the  study  as  a  whole  should  be  viewed  as  descrip- 
tive  and  explanatory  rather  than  as  an  exactingly 
controlled  experiment •     Viewed  in  this  light  the 
study  offers  a  number  of  provocative  leads  for 
needed  investigation." 

2.  Additional  research  is  necessary  before  concluding 
that  specification  Of  audience  and  rhetorical 
purpose  does  not  affect   the  quality  of  a  student's 
performance  on  a  timed  writing  test.     **Clearly  what 
is  needed  is  additional  research  on  pairs  of  prompts 
in  which  the  difference  in  audience  specification 

is  made  more  evident.     The  failure  to  fltid  dif- 
ferences in  the  quality  of  writing  between  these 
two  versions  may  have  been  due  to  the  fact  that 
they  both  represent  middle  range  examples  on  the 
continuum  of  audience  specification." 

3.  The  failure  to  find  topic-related  differences  in 
the  quality  of  writing  may  have  been  related  to 
the  fact  that  this  was  only  a  single  writing 
assessment  episode^  and»  as  sucht  provided  no 
opportunities  to  gain  insights  into  whether  this 
particular  performance  was  truly  representative 
of  these  students*   usual  writing  practices. 
Observing  the  same  students  over  several  writing 
tasks  would  undoubtedly  reveal  pa t terns--8uch  as 
"whether  some  kinds  of  tasks  consistently  require 
more  planning  time  than  others. . .and  whether 
planning  consistently  produced  better  products." 

4«     Whether  degree  of  audience  specification  can 
serve  to  trigger  more  and/or  faster  mental 
associations  or  responses  in  the  student-writer 
has  important  implications  for  evaluation  of 
instructional  programs  and  for  research  seeking 
to  assess  performance  differences  among  able  as 
well  as  less  able  student  writers. 

5.  What  are  the  effects  of  time  constraints  on  quality 
of  writings  on  forms  selected^  on  amount  of  planning 
time»  amount  of  revision^  etc^? 
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Part  II:  Comparison  and  Analysis  of  Rater 
Response  to  the  Anchor  Papers  in 
the  Writing  Prompt  Variation  Study 

Catharine  Keech  and  Mary  Ellen  McNelly 


Problem 


1*     This  part  of   the  study  represents  an  attempt  to 
"identify  factors  in  writing  test  design  which 
may  contribute  to  the  instability  of  student 
performance  on  direct  measures  of  writing"  by 
comparing  "student  and  teacher  evaluative 
criteria  in  scoring  student  essays."  Two 
focuses  of  study: 

a)  An  examination  of   task  interpretations  of 
student  writers »   reveal  ted  through  their 
discussion  of  the  qualities  of  the  anchor 
papers  during  a  training  session  tha t  pre- 
pared them  for  a  holistic  scoring  of  their 
own  papers ; 

b)  Comparisons  of  differences  between  students 
and  expert  raters  in  terms  of  interpretations 
and  evaluative  criteria* 

2.     This  part  of   the  study  also  included  an  attempt 
to  explain  the  apparent  lack  of  congruence  in 
task  interpretation  between  students  and  testers 
(or  teachers)  which  leads  to  apparent  failure  in 
student  performance^  as  determined  by  teacher- 
raters*     The  problem  is  approached  through  two 
primary  questions : 

a)  Do  writers  and  raters  have  the  same  perception 
of  what  the  task  is? 

b)  Do  writers  and  raters  have  the  same  perception 
of  what  constitutes  a  successful  completion  of 
the  task? 


Outcomes 


1.  A  major  finding  was  "that   some  students  do  in  fact 
interpret  the  writing  task  in  ways  which  differ 
from  teacher-rater  task  interpretation." 

2.  Differences  between  the  three  groups  (student 
raters^  novice  teacher  raters^  expert  teacher 
raters)   in  the  Inter-rater  reliability  could  be 
accounted  for  by  examining  the  nature  of  and 
attitudes  toward  the  training  for  holistic  scoring 
and  the  initial  defining  of  scoring  categories* 
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3.     Versitnilitude  on  writing  assessment  tests  ©ay 
actually  be  penalized  by  teacher-raters  because 
such  "real  writing"   (e.g.,   a  letter)   is,   in  fact, 
different  fro©  "real  English  the©es"   (or  "school 
writing").     "The  conflicting  fra©es  of  reference 
provided  by  the  8i©ulated  life-like  co©©unicat ive 
situation  and  the  actual  writing  context — a  class- 
roo©  test  occasion — can  be  successfully  integrated 
only  by  the  ©ost  able  student." 

C •     Issues  and  Reco©©enda t ions 

1.  One  difficulty  encountered  was  that  the  process 
of  choosing  anchor  papers   to  define  a  scoring 
range  —  a  critical  step  in  «ny^  holistic  assess- 
©ent--require8  the  juggling  of  so©eti©e8  contra- 
dictory constraints. 

2.  Lack  of  «gree©ent  arnon^  the  three  groups  of 
raters  could  be  explained,   in  part,  by  an  exa©i- 
nation  of  the  values  and  criteria  they  expressed, 
with  regard  to  particular  anchor  papers,  during 
the   training  sessions. 

3.  An  issue  raised  which  requires  further  study  is 
the  differences  between  responses  to  typed 
versions  of  papers   (used  in  reading  and  rating 
anchors  during  the  training  session)  and  original 
handwritten  versions  of  papers  used  during  the 
actual  reading  of  the  test  papers. 
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STUDY   3.        BEYOND  THE  FIELD   TEST  OF  WRITING  PROMPTS: 
A  PRELIMINARY  INVESTIGATION  OF  THE  NATURE 
OF  RESPONSE  VARIATION  TO  SELECTED  FIELD- 
TESTED  PROMPTS 

Part  I:     The  Effects  of  Assessment  Prompt 

and  Response  Variables  on  Holistic 
Score:     A  Pilot  Study  and  Validation 
of  an  Analysis  Technique 

Charles  Kinzer  and  Sandra  Murphy 

A.  The  Problem 

1*     In  examining  writing  task  variables  in  holistic 
writing  assessments  t   the  focus  in  this  part  of 
the  study  is  on  possible  relationships  between 
assessment  prompts^  written  products*  and 
holistic  scores*     The  study  represents  a  pre*- 
liminary  effort  to  determine  ways  to  identify 
unintended  effects  of  prompt  and  response 
variables  in  relation  to  holistic  scores. 

2.     Two  major  goals  of  this  study: 

a)  To  investigate  prompt  effects  and  response 
variables  which  might  result   in  discrepancies 
in  holistic  scores* 

b)  To  validate  an  analysis  system  for  looking 
within  specific  prompts* 

B .  Outcomes 

1.  The  results  of   the  study  support  the  viewpoint 
that  "effects  of  writing  prompts  are  related *in 
student  responses  and  influence  holistic  scores*** 

2.  The  study  found  that  **task  demands  can  be  related 
to  score*   and   that  an  analysis  system  modeled 

on   the  one  employed  herein  can  be  valuable  in 
determining  how  well  students  respond  to  a  prompt* 
and  in  determining  how  difficult  a  prompt  is*''* 

C*     Issues  and  Recommendations 

*'The  refinement  of  procedures  for  the  identification  and 
analysis  of  effects  of  writing  prompts  is  a  necessary 
step  toward  devising  ways  to  help  teachers  and  evaluators 
improve  the  formulation  of  prompts  for  writing  assessment 
and  instruction*     Given  the  widespread  use  of  direct 
measures  for  evaluating  writing  competence*  every  effort 
should  be  made  to   control  for  unintended  effects  in  the 
prompts  used  in  these  assessmen tsT" 
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Part  II:     A  Field  Test  of  Eight  Prompts:  Santa 
Clara  County  Writ  in g  Assessment 


Gerald  Camp 


A.     The  Problem 


The  goal  of   this  part   of  the  study  was  to  obtain  data 
on  the  performance  on  each  of  the  eight  prompts — developed 
for  use  in   the  Santa  Clara  County  Writing  Assessment — in 
typcial  county  schools  identified  as  low,  mid^range,  and 
i  high  based  on  the  performance  of  students  on  other 

measures  of  writing  ability. 


B.  Outcomes 


1.  Teacher-readers  were  asked  questions  after  completing 
the  scoring  of  the  papers.     The  first   two  questions 
asked  whether  the  teachers  felt  that  any  of  the 
prompts  produced  noticeably  weaker  or  noticeably 
stronger  writing  than  others.     The  responses  to 
these  questions  showed  very  little  agreement  among 
the  teacher-readers  as  to  which  of  the  eight  prompts 
were  less  and  which  were  more  successful. 

2.  Almost  all  of  the  prompts  were  successful  in 
producing  the  full  range  of  writing  quality 
desirable  in  such  a  sampling. 


C.     Issues  and  Re commend a t ions 


"Writing  short  timed  essays  on  subjects  students  have 
had  no  opportunity  to  prepare  for  requires  somewhat 
different  skills  from  those  needed  for  the  usual , school 
writing  assignments.     It  seems  only  reasonable,  there- 
fore,  that   students  should  be  given  some  training  in 
these  skills  before  being  required  to  demonstrate  their 
compe  tency  with  them. 

The  best  training  for   tests  of  this  sort  is  frequent 
in-class  practice  with  the  kinds  of  topics  which  will 
be  used  for  actual  testing.     Students  can  learn  how 
their   test  papers  are  judged  if  they  also  learn  to 
score  each  other*s  papers  themselves  using  the  holistic* 
method." 


ERLC 


33 

{  20 


STUDY   4:     A  STUDY  OF  THE   CONSTRUCTION  OF  THE  MEANINGS 
OF  A  WRITING  PROMPT  BY   ITS  AUTHORS »  THE 
STUDENT  WRITERS,   AND  THE  RATERS 

Part  I:     Overview  of  Goals  atid  Procedures  of 
the  Study 

Karen  Caroll     and  Sandra  Murphy 

Part  II:     The  Writing  Prompt:     Differences  In 
Expectations  Among  Participants  in 
a  Writing  Test  Episode 

Charles  Klnzer,  Karen  Carroll  and 
Sandra  Murphy 

Part  III:  The  Writing  Prompt:  The  Process  of 
Interpretation  and  Performance  That 
It  Elicits  in  Student  Writers 

Sandra  Mutphy,  Karen  Carroll,  and 
Ann  Robyns 

A.  The  Problem 

1.  The  problem  of  student  misunderstanding  of  teacher 
intention  in  comprehending  the  wilting  assessment 
is  more  grave  and  lasting  in  lar^e-scale  writing 
assessment  episodes,  in  which  opportunities  for 
feedback  do  not  exist  once  the  w]^|iting  has  begun, 
than  in  a  classroom  setting  in  which  the  teacher 
is  accessible. 

2.  Investiga.t ion  of   the  student  interpretation  of 
prompts  involved  interviewers  recording  the 
interpretations  .of  student-writers  prior  to  , 
writing  and  immediately  afterwards  in  the 
assessment  episode. 

3.  Investigating  "teacher  intention"  involved  asking 
teachers,  while  in  the  process  of  constructing 
the  prompt   to  be  used  in  the  assessment,  to 
verbalize  their  expectations  of  students*  responses. 
Later  other  teachers  predicted  student  interpre- 
tations and  prompt -based  outcomes . 

B.  Outcomes 

1.  Data  in  the  study  indicated  that  in  this  assess- 
ment,  in  spite  of  careful  construction  of  the 
prompt,  expectations  of  evaluators  differed  from 
those  of  studen t -writer s ;   in  addition,   the  data 
indicated  that  implicit   task  demands  may  have 

an  effect  on  the  way  students  *  papers  are  evaluated 
and  ranked. 

2.  *'Naive  scorers'  prompt-based  expectations  were 
extremely  similar   to  those  expressed  by  the  authors 
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of   the  prompts." 


"Teachers  and  students  often  perceived  task 
demands  differently.     One  major  difference  was 
that   the  prompt  authors  lent  weight  to  all  parts 
of  the  prompt,  while  some  of   the  student-writers 
focused  on  only  a  few  selected  aspects  of  the 
prompt." 

"In  the  larger  group  of  students,   the  results 
indicated  that  students  who  rated   the  prompt  as 
easy  to  read  and  understand   tended   to  receive 
higher  holistic  scores.     High  scores  were  also 
associated  with  agreement   that  it  was  important 
to  explain  why  the  subject   chosen  was  a  problem, 
and  that  it  was  imp or  tan t   to  suggest  a  solution 
to  the  problem.     The  results  also  indicated  that 
students  who  agreed  with  the  statement  that  it  was 
important   to  explain  how  the  solut ion/correction 
of   the  subject  chosen  would  improve  Central  High 
tended  to  receive  higher  scores.     Though  the 
measures  of  associations  are  low,   they  do  support 
the  view  of   the  impor tance  of  the  effects  of 
writing  task  comprehension  in  a  writing  test 
solution. 

"An  underlying  assumption  was  that   the  student 
accesses  information  in  the  piocess  of  reading 
the  prompt  which  he  or  she  then  uses  in  composing 
a  written  response.     The  construction  of  an 
interpretation  of   the  prompt  is  thus  viewed  as 
an  integral  and  important  part  of   the  composing 
process.     In  the  process  of   interpreting  the 
prompt,   the   student  may  selectively  focus  on 
some  segments ,   ignore  other s,   and  interpret 
the  ones  assimilated  differently  than  do  the, 
evaluators  in  the  assessment.     Thus,   the  student 
may  "construct"  an  interpretation  of  the  prompt 
which  differs  from  the  interpretation  of  other 
participants   in  the  assessment . " 

"In  order   to  determine  how  students  used  their 
initial  Interpretations  in  their  written 
compositions,   segments  of   the  oral  interviews 
were  compared  to  segments  of   the  written 
compositions .     Both  similarities  and  differences 
were  found  be  twee n  the  oral   language  of  the 
interviews ,   and  the  written  language  of  the 
students'    compositions.     Basically   the  similarities 
were  found   in  content,  while  the  differences 
were   found  in  rhetorical  function." 

"The  study  suggested  that  context  plays  an 
important  role  in  both  the  reading  and  the  writing 
process.     However,   the  issue  was  not  simply  one 
of  ''knowledge"  or   lack  of   it;   rather,  the 
issue  was  whether  or  not  a  context  should  be 
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STUDjr  5:      HOLISTIC  ASSESSMENT  AS  LONGITUDINAL  MEASURE 

OF  STUDENT  GROWTH:  INTERPRETING  STUDENT  o 
TASK  CONSTRUCTIONS 

Part  I:     The  Theoretical  Model 

Catharine  Keech 


A.  The  Problem 

1.  '*Why  do  performance  scores  an  writing  tests  so 
often  fail  to  improve  iti  neat  positive  intervals 
for   individuals  and  groups ?*' 

2.  One  aspect  of  performance  variation  is  the 
phenomenon  of  apparent  regression-- the  U-shaped 
learning  curve --during  the  acquisition  of  new 
skills  and  performance  strategies* 

3.  Another  aspect  of  performance  variation  ia  "the 
ill-defined  nature  of   the^ writing  task^  which 
allows  the  student  writer   to  determine^   to  a 
large  extent^   the  difficulty  level  of  what 
attempts   regardless  of  tS^e   tester's  intention." 

B .  Outcomes 

1.  "Apparent   regression"  may  simply  be  one  manifesta- 
tion of   the  non-linear  nature  of   improvement  in 
developing  writers.     Errors  committed  by  writers- 
in-transi tion   (i.e.,   developing  writers  who  are 

in  the  process  of  learning  a  more  advanced  writing 
strategy)   don't  necessarily  indicate  that  they 
are  less  able  performers  than  before  the  new 
learning  began.     Rather,   these  errors,  when  seen 
in  context,  may  be  signalling  movement   toward  a 
higher  level  of  compe tency .     That  is,  wr iters- in - 
transition  can  be  perceived  as  practicing  their 
newly-learned  writing  strategies. 

2.  In  some  writing  assessments   (or  writing  assign- 
ments)  the  complexity  of  the  cognitive  task  ^ 
demanded  may  distract  tht^  writer's  attention  away 
from  rhetorical  f ea tures-^even  those  that  the 

wr  iter  has  previously  mastered. 

3.  "How  the   student  defines  the  rhetorical  task  at 
the   text   or  discourse  levl^l,"  how— and  whe^ther-- 
the  student  performs  whole^^text  planning  (£ii4t  Is, 
conceptualizing  an  "artifacrC^  that  he  or  shej  is 
about  to   create)  can  Contribute   to  developmental 
unevenes  8 . 

4.  Unexpected  responses  to  writing  tasks  by  student- 
writers  have  inevitably  occurred  no  matter  how 
carefully  test  makers  have  tried  to  co<itrol  for 
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provided,   and  when.     If  a  prompt   is  Intended  to 
be  open  9   prompt  authors  should  avoid  obscured 
contexts  which  may  lead  some  readers   (such  as 
many  of   the  students  in  this  study)  io  attempt  a 
re  cons true  t ion  of  the  authors'   intended  frame 
of  reference*     If  a  context  of  prior  knowledge 
is   considered  to  be  important^  ast  of  course^ 
it   is  in  many  assessment  situations*   then  prompts 
should  clearly  specify  what  this  context  is," 

8.  "When  students  have  only  one  chance  to  write  and 
are  being  Judged  on  this  single  perf  orihance » 

It   is  especially  Important  that   they  have  optimal 
conditions  for   their  performance*   and   that  there 
be  no  mismatches  of  either  topic  or  task." 

9.  "Various   interpretations  were  made  of  prompt 
segments*   and  these  interpretations  were  related 
to  rhetorical  strategies  adopted  by  the  students 
in  composing  a  response." 

10.     "The   first  step  for  both  the  student  and  the 

evaluator   in  a  writing  assessment   is  the  reading 
and   interpreting  of  a  prompt*     Thus*   the  features 
of   a  prompt  which  influence  interpretation  are  of 
critical  importance  for  both  the  reliability 
and  the  validity  of  our  assessment  measures." 

Issues   and  Recommendations 

1.  Further  study  is  needed  of  "possible  within-top ic 
salience*"  or   the  possibility  that  certain  individual 
words   in   the  prompt  might  cue  certain  individual 
students  more  strongly   (because  Cf  the  effect  of 

the  student's  prior  knowledge  on  his  or  her  response 
to   certain  words). 

2.  Dij&agreemen t  emerged  between  students  and  teachers 
as   to  whether  or  not  the  writing  produced  by  the 
prompt  would  give  a  good  indication  of  writing 
ability* 

3.  "The  study's  findings  have  serious  implications 
for   current  practice  in  assessment*   since  mis* 
matches  of   the  kinds  described  can  result  in 
distortions  of   the  ways  individual  students  are 
judged." 
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such  responses. 

5.     Instead  of  continuing  to  narrow  task  constraints 
In  the  writing  assignment  as  a  way  to  control  for 
unexpected  outcomes,  evaluators  should  attempt  to 
describe  what   students  do  differently  from  one 
writing  occasion  to  the  next.   Interpret  that 
*  difference,   and  attempt  to  measure  the  development 
It   Implies.      (This   Is  what  was  attempted  on  the 
Dfak6  longitudinal   study,   using   the  typology 
described  here.) 

• 

Issues  and  Recommendations 

1.  "Empirical  research  Is  needed  to  help  discover 
why  a  student  construes  a  task  as  she  does: 
what   Is  the  range  of  options  she  has?     How  does 
she  want  her   text   to  function  and  why  did  she 
set  that  goal?     What  strategies  will  she  choose 
to  reach  her  goal?     Is* she  an  oldtlmer  or  a 
newcomer  to  her  chosen  strategy?     What  awarenesses 
of   text  grammar  appear  to  Influence  her  whole 
text  planning  or  her  sen*  ^nc*  level  planning? 

How  Is  the  onset  of  teAi;  le^el  planning  related 
to  Increasing  awareness  of  discourse  schemas? 
The  discourse   typology,   refined  and  clarified, 
may  be  of  some  use   in  explrrlng  ther»e  wuestion*:, 
questions  which  way  help  us  discover  why  an4l  how 
students  change  tre  difficulty  of  what  they 
attempt,   even  when  a  tester  may  interd  task 
demands  to  be  parallel.   These  qu^Htions  4. ay  also 
help  us  define  task  complexity  in  new  v-ays, 
allowing  more  precise  evaluation  of  changes 
or  groyth  in  composing  ability." 

2.  "Much  greater  clarification  of  discourse  features, 
and  of  the  role  played  by  text  level  expectations 
of  both  writers  and  readers,   seems  needed  before 
it  will  be  possible  to  measure   improvenent  in 
composing  abilities*'t 
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Part  II:     The  Model  In  Practice 


Ellssa  Warantz  and  Catharine  Keech 


The  Problem 

1.  A  longitudinal  examination  of  ''the  extent  to 
which  students  at  tempt  to  do  different  kinds 

of  writing,   either  spontaneously  or  In  response 
to  strongly  suggestive  features  of  the  writing 
ssslgnment,    in  a  series  of  annual  holistic 
writing  tests  taken  during  their  ninth  to 
twelfth  years  In  high  school."     The  hypothesis 
was   that   the  differences  in  what   the  student 
attempted  —  the  task  construction — might  help 
account   for   "unexpected  directions  of  change  in 
"student  performance  scores  from  year  to  year." 
Results  of   this  research  would  contribute 
toward  an  understanding  of  "the  seeming 
instability  of  student  performance  from  one 
test  occasion  to  another,"  dlrectltig  attention 
to  the "developmental  features  of   text  design 
and  task  analysis  ii^hich  may  influence  the 
quality  rating  of   student  essays  In  Seemingly 
paradoxical  ways ;   I.e.,  when  student  test  scores 
seem  to  deteriorate  over   time,   rather  than 
improve . " 

2.  Although   test  makers  can  control  the  set  of 
constraints  and  options  expressed  or  Implied 
in  the   text  of  the  writing  assignment  (the 
given  task)   and  thus  create  parallel  task 
demands  on     ^  subsequent  occasions, 
they  are  less  able  to  control  the  student*s  - 
construal  of   the   task,  which  determines  the  set 
of  constraints  actually  honored  and   the  options 
actually   chosen  by  the  student   in  writing  (the 
constructed  task) . 

3.  Prior   to  any  effort  at  measuring  the  development 
of  writing  ability,  what  Is  needed  Is  "a  method 
of  comparing  what   studen^ts  actually  do  and  how 
they  do  it,   from  one  occasion   to   the  next." 
Therefore,   the  first  step  In  this  research  was 
"the  derivation  of  a  typology  of  task  construction 
which  would  adequately  account   for   the  range  and 
variation  of  writing  produced  by  students,  as 
well  as  those  texts  traditionally  classified  In 
terms  of  literary  'genres.'" 
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Outcomes 


!•     The   typology  derived  was  applied  to  a  sample 
of  student  papers  and  the  resulting  data 
provided  a  way  to  "compare  individual  student 
performance   (in  terms  of  task  construction  and 
score)   to  the  favored  task  construction  and  mean 
score  of  all  students  in  a 'given  topic/year." 

2.  The  most  frequent  source  of  discrepancy  in 
coding  of   functions  and  strategies   (a  procesf 
that   the  teacher-raters  were  trained  to  carry  out 
with  these  papers)  appeared  in  papers  which, 
according  to  the  raters,   seemed  to  contain  some 
combination  of  description  and  evaluation  in 
function  and/or  strategy. 

3.  Discrepancies,  along  with  the  general  ambiguity 
in  the  final  coding  of  such  papers,  seemed 
attributable  to  two  factors:     (1)   ^opic  effect, 
and   (2)   the  "bleeding  strategy  effect'*  on 
rater  perceptions. 

4.  Another  relatively  large  group  of  papers  for 
which  the   typology  could  not  adequately  account 
were  coded  as  "unclassif iable"  in  terms  of 
function  or  strategy  or  both.     These  papers 
"appeared   to  fulfill  mote  than  two  functions 
and/or  strategies  simultaneously   (and  therefore 
contained  no  clearly  defined  primary  function  or 
strategy),   or  they  fulfilled  a  social  or  phatic 
function. . .which  overrode  any  rhetorical  function 
included  in  the  typology." 


Issues  and  Recommendations 

1,  "Only  after  we  are  able  to  describe  what  the^ 
student  actually  produces,   in  response  to  test 
prompts,  will  we  be  able  to  infer   the  student's 
conceptualization  or  construction  of  the  writing 
task.     Then  we  will  be  able  to  generate  and  test 
hypotheses  about  the  relative  ilevel  of  difficulty 
of  what  is  attempted.     That  research  should,  in 
turn,  make  possible  a  proper  investigation  of 
the  development  of  writing  skills,  by  allowing 

us  to  distinguish  between  a  student's  mastery  of 
old  tasks  and  his  novice  attempts  to  produce  new 
tasks.     Further,  such  research  may  ultimately 
allow     researchers  to  predict  which  kinds  of 
tasks  are  best  learned  first,  which  follow,  and 
what  the  various  spontaneous  task  constructions 
employed  by  students  na^y  indicate  about  a  given 
student 's  particular  level  of  development . " 

2.  After  determining  which  task  constructions  were 
favored  for  a  given  grade  within  a  given  topic/ 
year,  further  analysis  would  reveal  whether  a 


student's  use  of  an  **unf avored**  task  construction 
in  a  given  year  night  have  affected  that  student's 
score. 
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PART  I:    SOURCES  OF  KNOWLEDGE  FOR  DESIGNING  WRITING  TEST  PROMPTS 

OVERVIEW  OF  THE  STUDY 

Often  the  first  queitibn  ««ked  in  ichooli  embarking  on  an 
assessment  of  writing  ia.   "What  topic  shall  we  use  and  how 
should  we  word  it?"     An  enterprising  Coordinator  of  Competency 
Assessment  for  a  California  high  school  offers  the  following 
advice  in  an  article  on  "Organizing  for  Holistic  Scoring  in 
California  English  (Paulin,  1980): 

Develop  a  list  of  promising  tapics— a«  many  as  you 
can  find,  borrow,  or  lift  from, other  assessments. 
Write  up  directions  to  students  for  each  topic. 

Is  ther#  no  batter  advice  to  offer  prospective  evaluators 
than  simply  to  pillage  from  previous  tests?     Is  there  some 
rational  procedure  for  the  generation^of  writing  test  prompts? 
Are  there  systematic  and  theoretically  sound  rules  for 
designing  appropriate  writing  tasks  for  local,  state,  or 
national  assessment*  of  writing? 

These  are  important  questions  to  seek  answer's  for,  be- 
cause it  is  the  writing  test  prompt  that  initiates  a  whole 
process  culminating  in  a  judgment  of  writing  competence  that 
presumes  to  say  something  meaningful  about  the  writing  "Fa- 
ience of  the  student  in  delation  to  the  instruction  received. 
Thus,  the  conception  of  a  writini  task  implies  a  model  of 
discourse  forms  and  functions  and  a  theory  of  the  nature  of 
the  writing  process.     I«ismuch  as  the  prompt  »ust  stand  in 
dynamic  relation  to  the  piece  of  writing  it  initiates, 
prospective     evaluators  presumably  would  desire  to  Learn  how 
the  various  properties  of  writing  prompts  might  affect  the 
perforiiiance  of  writers  in  both  intended  and  unintended  ways. 
So  if  our  California  Coordinator  of  Competency  Assessment 
were  to  look  to  those  authoritative  sources  that  school 
people  are  so  often  adjured  to  consult,  what  useful  knowledge 
might  she  uncover? 

The  Research  Problem.     The  principal  research  purpose 
governing  the  work  of  all  dimensions  of  this  NIE/BAWP  Writing 
Assessment  Project  is  to  increase  our  knowledge  of  the  P"?"" 
ties  of  writing  test  prompts  and  the  compositions  they  initi- 
ate.    There  is  ample  evidence  in  daily  experience  to  enable 
us  to  conclude  that  particular  prompts  do  not  perform  as 
uniform  stimuli  across  populations  to  guarantee  the  activa- 
tion of  expected  levels  of  thought  and  forms  of  discourse. 
This  investigation  asks  the  questions: 
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1.  What  elements  in  the  content  and  structure  of 
writing  test  prompts  influence  the  production 
of  compositions  during  writing  assessments? 

2.  What  makes  a  prompt  ''good"  for  eliciting 
writing  samples  to  be  rated  under  conditions 
of  "holistic  scoring"  procedures? 

3.  How  are  "good"  writing  prompts  generated  and 
tested  systematically? 

.4.     What  changes  occur  in  maturing  students* 

responses  to  writing  prompts  in  successive 
annual  assessments*  and  what  do  these  changes 
reveal  about  how  we  should  be  assessing 
developing  abilities  in  writing? 

5.     What  sources  of  knowledge  do  we  have  to  guide 
current  practice  in  designing  effective 
writing  prompts? 

This  chapter  addresses  questions  2»   3,  and  5,  emphasizing 
primarily  a  description  of  the  sources  of  knowledge  from 
theory,  research  and  practice. 

William  Irmscher,  currently  president  of  the  National 
Council  of  Teachers  of  English,  observes,  "...it  is  hardly 
possible  to  discover   new  ground  if  one  does  not  even  know 
what  is  established  territory."     Therefore,  the  place  to 
begin  to  seek  answers  to  the  first  two  questions  above  is 
with  a  review  of  the  research.     To  date,  there  appears  to 
have  been  only  me  other  comprehensive  review  of  research  on 
writing  topics  (Hoetker,  1980),  described  more  fully  later. 
The  review  envisioned  here  will  reach  out  for  new  ground  not 
covered  by  Hoetker,  especially  in  the  realm  of  ret respect ive 
generalisations  collected  from  prof •ssionals— taachers , 
scholars,  researchers,  professional  evaluators.  Taken 
together  their  collective  practical  wisdom  coalesces  into  a 
kind  of  "theory  of  practice"  to  be  discussed  later.  The 
more  conventional  sources — psychome tries ,  for  example — also 
are  consulted. 

Following  the  extended  review  of  the  literature,  a 
second  phase  of  this  study  of  "who  knows  what  about  designing 
good  writing  prompts"  will  describe  a  procedure  whereby  we 
circulated  a  draft  of  a  report  on  practices  in  designing 
writing  test  prompts  among  two  panels  of  reviewers:  one 
was  a  cadre  of  teachet-^consul tants  from  the  Bay  Area  Vritiprg 
Project  and  other  writing  instructors;  the  other  was  a  grorup 
of  writing  researchers  and  professional  test-makers.  The 
responses  of  the  two  panels  are  collated  and  reported  later 
in  this  chapter,  and  the  revised  report  on  practices  in 
designing  writing  test  prompts  constitutes  Part  II  of  this 
chapter. 
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Limitations  of  the  Study.     Because  the  Bay  Area  Writing 
Project  has  pioneered  the  dissemination,  through  its  many 
National  Writing  Project  sites,   of  a  "holistic  writing  assess- 
ment" procedure,   this  study  is  limited  to  an  examination  of 
those  kinds  of  prompts  which  may  serve  the  purposes  of  this 
model.     Thus,   this  study  excludes  thewell-regarded  "primary 
trait"  system  of  the  National  Assessment  of  Educational 
Progress.     The  primary  trait  system  requires  uniform  specifi- 
cation of  a  set  of  task  constraints  in  order  to  "evaluate  the 
capacity  to  write  for  precisely  defined  purposes,"  according 
to  the  most  recent  NAEP  Procedural  Handbook,   1978-79  Writing 
Assessment.     A  fuller  discussion  of  this  model  is  provided 
later  in  this  chapter,  but  the  type  of  writing  "exercise 
derived  f-rom  the  primary  trait  model  is  not  treated  in  any 
great  detail.     Other  scoring  systems — analytical,  feature 
analysis,  essay  scales — which  may  also  entail  certain  con- 
siderations in  the  design  of  writing  prompts  arc  not  con- 
sidered here. 

Neither  is  the  principal  instrument  of  ongoing  writing 
assessment,   the  class  writing  assignment,  considered  here. 
The ''class    writing  assignment  1b  one  of  the  areas  of  investi- 
gation covered  in  another  BAWP/NIE  project:     "Research  on 
Effective  Writing  Instruction  in  Inner  City  Secondary  Schools. 
Class  assignments  also  frequently  are  the  subject  of  project 
publications  such  as  the  National  Writing  Project  series  on 
Writing  Teachers  at  Work,  e.g.,  Ray  Marik*.  Special  Education 
Students  Write:     Classroom  Activities  and  Assignments  (Univer- 
slty  of  California,  Berkeley:     NWP ,   1982).     There  is  also 
the  series  of  Classroom  Research  Studies  .done  by  three  class- 
room teachers— Patrick  Woodworth,  Rebekah  Caplan,  and  Stephanie 
Gray  in  association  with  Catharine  Keech  as  the  research 
design  consultant.     These  publications  are  listed  and  des- 
cribed in  a  Publications  for  Teachers  list  appended  to  this 
chapter.     It  is  necessary,  however,   to  take  a  moment  to 
distinguish  between  the  nature  of  class  writing  assignments 
and  school  writing  tests. 

Class  Writing  Assignments  and  School  Writing  Tests 
Distinguished.     Because  it  has  been  frequently  suggested  that 
our  focus  on  the  study  of  writing  assessment  prompts  be 
expanded  to  include  classroom  writing  assignments  as  well,  it 
is  necessary  to  distinguish  carefully  between  the  character 
of  these  two  types  of  occasions  for  writing.     The  prompt  that 
appears  in  a  testing  context  must  function  autonomously:     it  ^ 
permits  no  aid  in  Interpretation  from  the  proctors  of  the 
•vent:   it  offers  no  specific  preparatory  context  or  motivating 
impulse  other  than  the  command  to  write.     The  conditions  for 
the  participants  are  carefully  controlled  In  order  to  be 
depersonalized,   formal,   standardized.     The  underlying  psycho- 
metric assumption  upon  which  the  stability  of  such  measurement 
relies  is  th*  belief  in  the  uniformity  of  the  stimulus  proper- 
ties for  each  student  responding  to  the  prompt.     Each  head 
bent  over  the  page  is  presumed  to  be  getting  the  same  message 
to  direct  his/her  writing  performance. 


34 


By  contrast  •  the  prompts  (pr  writing  assigniaent s)  that 
appear  in  a  classroom  instructional  context  function  in  a 
setting  that  encourages  negotiat^ion  of  interpretation.  The 
teacher  may  offer  a  good  deal  of  guidance^  making  clear  the 
expectations  as  well  as  the  possibilities  of  the  assignment. 
The  teacher  may  orient  the  class  to  the  area  of  experience* 
to  be  dravA  upon  and»   through  discussion^  may  enlarge  the 
ground  of  shared  knowledge.     The  teacher  may  give  example-s  ^ 
of  appropriate  and  inappropriate  s trategies  for  discovering 
and  structuring  the  essential  material  of  the  composition. 
And»   if  the  writing  occurs  in  class*  the  teacher  can  assist 
in  repairing, any  of  the  breakdowns  that  may  happen  if  initial 
attempts  to  progress  through  the  writing  assignment  fail.  The 
process  is  personal»  more  informal*  onstandardized  and  keyed 
to  the  proposition  that  differential  interpretations  of  the 
prompt  are  inevitable  but  also  reconcilable  through  negotiation. 
A  formulation  of  major  distinctions  between  the  use  of  writing 
prompts  in  testing  occasions  and  in  teaching  occasions  follows: 


PARTICIPANTS 
AND  PROCESS 


WRITING  ASSIGNMENT  IN 
A  TEACHING  CONTEXT 


WRITING  PROMPT  IN 
A  TESTING  CONTEXT 


1.  Population: 

2.  Leader: 

3.  Assignment 
Text : 

4.  Interaction; 

5.  Writing"^ 
Process: 


6.  Expects* 
tions 

7.  Evaluation: 


Class  members 

Teacher 

Negotiable 

Allowable 

Interpersonal 

Reciprocal 

Informal 

Known 

Flexible 

Contingent 
Graded 


Assembled  students 

Unknown  proctor 
Autonomous 

Prohibited 

Depersonalized 

One*way 

Standardized 

Partly  implicit 

Fixkd  standard 
Ranked 

Scored 


The  two  types  of  assignments  must  meet  different^  criteria  to 
perform  their  unique  functions.     Thus  assignmenta  for  teach- 
ing and  testing  are  not  easily  interchangeable.     Keeping  these 
sorts  of  distinctions  in  mind»  we  shall  now  examine  the 
several  sources  of  knowledge  we  have  available  to  inform 
practice  in  designing  writing  test  prompts.     The  chief  sources 
to  be  consulted  in  the  following  section  are  psychdmetr ics » 
professional  wisdom»  discourse  theory*  and  research. 
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SOURCES  OF  KNOWLEDGE  FROM  THEORY  AND  RESEARCH 
\  FOR  DESIGNING  WRITING  TEST  PROMPTS 


In  the  sklllrul  framing  of  «  writing  tasky  more  than 
in  anything  el8e»  lies  the  fine  art  of  assessing  writing. 
In  the  writing  task  the  student  finds  the  spur  to  inagi- 
nation^   the  stimulus  to  thought^   the  guide  to  idea  struct- 
ture»  and  the  incentive  to  writing  performance.  The 
writing  task  set  in  an  assessment  literally  "prompts'"  the 
writing  performance  evoked.     Inasmuch  as  the  statement  of 
the  task  stands  in  dynamic  relation  to  the  piece  of  writing 
it  initiates^  examiners  presumably  would  desire  to  know 
how  particular  properties  of  tasks  might  affect  the, per* 
formance  of  writers  both  in  predicted^  and  in  unintended 
ways.     Given  the  long  history  of  the  use  of  written  exami- 
nations in  this  country^  one  night  expect  to  find  a  body 
of  knowledge  drawn  from  theory »  research,  and  practice 
which  guides  the  design  of  writing  test  questions,  assign- 
ments,  tasks,  or  prompts.     After  all,  the  written  exami-- 
nation  emerged  in  the  middle  of  the  nineteenth  century 
in  the  United  States  and  thereafter  became  the  dominant 
means  of  assessing  learning.     In  the  mid-thirties,  the 
very  term  examination  was  still  synonymous  with  written 
examination.     Thus,  a  report  on  the  subject  to  the  ^ 
Carnegie  Foundation  for  the  Advancement  of  Teaching  was 
called  Examinations  and  Their  Substitutes  in  the  United 
States   (Kandel,   1936).     In  this  report  the  "substitutes" 
for  "real"  examinations  were  the  "new  type  tests"  which 
provided  several  varieties  of- "objective"  questions. 

As  the  claims  of  superiority  for  the  "new  type"  test 
became  stronger,  efforts  were  being  made  to  formulate 
essay  questions  in  such  a  way  that  they  too  could  be 
marked  more  objectively  and  yield  more  reliable  results.  * 
But  as  Kandell  pointed  out,  "...the  more  accurately  and 
carefully  this  is  done,   the  closer  will  be  the  approxi- 
mation to  the  new  type  test  mnd  the  less  in  the  long  run 
will  be  the  real  educational  value  of  the  essay"  as  a 
matter  of  training  in  expression  (p.  85).     In  1936, 
according  to  Kandell,   the  leading  advocates  of  the  "new 
type"  test  were  not  yet  suggesting  that  it  should  replace 
the  essay  as  a  "form  of  training  and  practice  in  clear 
organization  and  logical  presentation  of  facts,  concepts, 
and  ideas"   (pp.  84-85). 

Nevertheless,  Kandell  did  forsee  that 

...there  is  a  real  danger  that,  because  of  the 
greater  reliability  and  validity  of  sound  objec- 
tive tests,  the  educative  value  of  writing  essays 
and  their  value  as  a  method  of  training  in 
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expression^   in  mssembllng  materials^   and  Id 
organizing  them  logically  may  be  neglected^ 
or,   if  not  neglected^,  be  relegated  to 
classes  in  English  composition  alone   (p.  68). 

Nov  that  the  "new  type"  objective  tests  have  become 
the  dominant  means  of  assessing^  writing  performance^  it 
is  difficult  to  imagine  the  time  when  written  examinations 
were  the  chief  means  of   taking  a  "daguerreotype  like- 
ness •••  of  the  state  and ^ condition  of  pupils'  minds" 
(Horace  Mann  quoted  in  Randellt   p.  26). 

Until  the  turn  of   the  century»  the  discussion  of  the 
properties     and  effects  of  written  examinations  was  still 
based  on  opinion  (Kandellt   1936»  p.   57).     But  as  the 
"scientific  movement"  in  education  began  to  grow»  pioneers 
in  educational  measurement  such  as  J.  Mckeen  Cattell  and 
Edward  L.  Thorndike  began  to  apply  statistical  methods  to 
the  study  of  written  examinations.     In  1905»  Cattell 
declared  that  "it  seems  strange  that^ no  scientific  study 
of  any  consequence  has  been  made  to  determine  the  validity 
of  our  methodst   to  ftandardize  and  improve   [written  exami- 
nations]."    In  this  same  paper »   Cattell  staked  out  the 
territory  that  ^as  to  become  the  contemporary  field  of 
psychometr ics  when^he  prophetically  argued 

It'  seems  scarcely  possible  to  determine  whether- 
students  are  fitted  for  a  college  course  by  means 

of  a  written  examination  To  devise  and  apply 

the  best  methods  of  determining  fitness  is  the 
business  of  the  psychological  expert,  who  will 
probably  represent  at  the  close  of  this  century 
as  important  a  profession  as  medicine,  law  or  church 
(quoted  in  Kandel,  p.   59).  s 

Cattail's  challenge  laid  the  foundation  for  subsequent 
study  of  the  reliability  of  essay  examinations  and  the 
development  of  their  "substitutes,"  objective  tests. 

Psychometr ics  as  a  Source  of  Knowledge 

Psychometr ics  has  dealt  with  a  restricted  range  of 
concerns.     As  Gilbert  Sax  defines  the  field:  "Educational 
measurement. . .requires  the  quantification  of  attributes 
according  to  specified  rules"   (1974,  p.   3).     These  rules 
deal  with  the  measurable  properties  of   tests  to  resolve 
issues  of  reliability,  validity ,  objectivity,  and 
standardization  in  the  administration,   scoring,  and 
interpretation  of  tests.     While  textbooks  in  measurement 
do  deal  with  test  construction  in  a  general  way,  their 
focus  tends  to  be  more  on  the  technology  and  methodology 
of  measurement  than  apon  the  substantive       properties  of 
the  items  or  the  essay  questions  in  a  test.     That  is  to 


fiay»  measurement  textbooks  provide  methods  of  applying 
statistical  approaches  to  the  appraisal  of  the  "behavior" 
of  writing  prompts  or  objective  test  items  during  field 
trials,   but   they  do  not  offer  mu'ch  beyond  the  most 
general  '  sor ts  of  principles  for  ^actually  designing  and 
wording  the  writing  tasks.     The  field  trial  provides  a 
self -evaluat ive  procedure  for  eliminating  faulty  essay 
questions  or  othei;;  test  items,  but  it  does  not  reveal 
what  has  gone  wrong  in  framing  the  item.     The  procedure 
is  simple:     .to  design  an  essay  examination  that  will 
distinguish  between  students  who  do  well  in  writing  and 
those  who  do  not,   the  examiners  simply  compose  several 
essay  questions,   administer   them  to  a  sample  of  ^the 
population  to  be  tested,  ab serve  whether  they  predict 
performance  along  th^  lines  expected,  and  eliminate  or 
change  questions  that  do  not  wo^fk..    The  essay  questions 
'that  remain  are  presumed  to  be  "meaningful/*  to  the 
larger  population  to  be  tested.     Psychome tr i cs  strives 
to  be  scientific,  but   the  procedures  used  to  c^otistruct 
and  validate  essay  questions  are  es^aentialTy  subjective 
and  scientifically  crude  in  the  sense  that  there  i«  no 
theory  of   information  processing  underlying  the  design 
of  writing  tasks.     Standardized  testing  procedures  pre- 
suppose  that   the  examiner  and  the  student  wr;Lter  sh'&re 
a  common  language  and  uniform  interpretations- of  tasks. 
But   the   functi^on     of   a  writing  task  as  a  stimulus  event 
cannot  be  understood  until   the  examiner,  understands 
clearly  how  the  constraints  of  language  use  and  interpre- 
tation operate  in  the  writing  assessment  €vent. 

Statistically  prienti^ed  evaluators  do.ti6t  seem  .to  be 
aware  that  the  psychometric  approach  to  assessments  6f 
ability  necessarily  presupposes  an  information  pr<ycessing  - 
model  of  communication.     Consequently,   psy chometricians 
have  not  attempted  to  study   the   Influence  of  "the  psy.dho- 
linguistic  and  sociol inguist ic  variables  on ^the  process  of 
interpreting  objective  items  and  essay  questions  us£d  in 
particular  writing  assessment  contexts.     Rather,  the 
emphasis   in  psychome  tries"  has  been  on  the  manipulat  ion 
and  treatment   of  products  in  measurement  schemes,  not  on 
the  understanding' of   the  cognitive  processes  that  might 
also  be  '  involved .     From  the  beginning,   psy chome tries  has 
been  in teries te d  in  concrete  ' ma tters  in  the  measurement  of 
an  educational  product.     As  defined  by  Edward  Thorn- 
dike   in   19 rS,   educational  measurement   is  based  upon  the 
following  theory: 

Whatever  exists  at  all  exists  in  some  amount . 
\      To  know   it   thoroughly   involves  knowing  its 

quantity  as  well  as'^lts  quality.     Education  is 
concerned  with  changes   in  human  beings;   a  change 
is  a  difference  between  two  cojidit  ions ;   each  of 
these  conditions   is  known  to  us     only  by  the 
products  produced  by  it — things  made,  words 
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spoken*   acts  performed  and   the  like.     To  measure 
any  one  of  these  products  means'^  to  define  Its 
amount  in  some  way  so.  tha t  ..compe  t ent  persons 
will  know  how  large   it   is»   better   than  they 
would  without  measurement.      (Thorndike's  1918 
statement   is   in  the  National  Society  for  the 
Study  of  Education,   Seventeenth  Yearbook, 
pp.    16-17,   as  quoted  by  Kandel,   p.  80). 

Given  this  notion  of  measurement ,   it   is  not  surprising 
that  Thorndike  and  other  early  leaders  in  measurement 
devoted  themselves  to  reducing  the  weaknesses  and  defects 
in  writ  ten  essay  types  of  examinations  by  devising  *'new 
type**   tests  which  yielded  results  that  were  objective  and 
were  considered  more  reliable  and  accurate. 

I.M.   Kandel,   reporting  to  the  Carnegie  Foundation  on 
the  state  of  knowledge   in  conducting  examinations,  observed 
that   investigations  had  been  **limited   in  the  main  to  a 
study  of   the  reliability  of  the  marking  of  examinations" 
(Kandel,   p.   3).     Thirty-five  years  later,   in  writing  the 
chapter  on  essay  testing  for  the   definitive  volume. 
Educational  Testing   (1971),   published  by  the  American 
Council  on  Education,  William  E.   Coffman,   a  researcher 
for  the  Educational  Testing  Service  and  the  College 
Entrance  Examination  Board,   could  still  confirm  Kandel's 
earlier  observation  about  the  dominance  of   issues  of 
reliability  in   the  study  of  essay  testing.     Coffman  says 
that  most  research  on  essay  testing  '*has  dealt   in  one  way 
.or  another  with  the  question  of   reliabilty**  (Coffman, 
1971,   p.   298).     The  most  recent  review  of  research  on  the 
effects  of  topics  on  student  writing  now  available  also 
suppprts   the  findings  of   the  two  previous  reviewers.  , 
After  conducting  his  comprehensive  review,  James  Hoetker 
(1981,   p.   A)   stated  that  "research  attention  has  been 
devoted  almost  entirely  to  issues  of  rater  reliability, 
ignoring  for  the  most  pprt  the  issue  of  validity  as  well 
as  the  other  two  sources  of  error  in  an  essay  examina t ion-- 
the   topics,  and   the  writer." 

Psychometric  literature  does  recognize   the  possibility 
of  variation  in  the  difficulty  of  writing  tasks,   but  it 
views   the  issue  broadly  and  non- analy t ically •     There  is 
seemingly  little  in  the  literature  of  measurement  (see 
Sax,  pp.   113-116)   and  educational  research  in  the  way  of 
empirically  determined  principles  for   the  formulation  and 
wording  of  4^say  test  questions.     Such  principles  can, 
however ,   be  found  in  the  literature  on  instrument  con- 
struction in  public,  opinion  research  on  the  wording,  of 
""questions,  for  interviews  and  questionnaires.  ^    (See  Payne, 
1951;   Oppejiheim,   1966;   Gordon,   1980).     Fot  e^catnple,  Payne 
and  McMorr  is   ( 1967 ,  p.    145),   in  aae^ditorial  notebiefore 
one  of   the  ^ele^tions  in  their  "Collection  of  readings  on 
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the  theory  and  practice  of  educational  neasur enen t ,  hava 
commented  as  follows:      "while  textbooks  dealing  with 
Instrument  construction  present  many  suggestions  for  Item 
writing,   the  empirical   evidence  demonst^tlng  the  worth 
of  such  apparently  sound  rules  Is  meageri^*'     The  few 
available  "rules"  appear  to  be  concerned  chiefly  with 
such  aspects  of   Item  construction  as  equal  length  answer 
alternatives  vs.   extra   long  answer  alternatives;  con- 
sistency vs.   Inconsistency  of  grammar  between  lead  and 
alternatives;    the  question  type  of  stem  vs.   the  Incom- 
plete statement   type  of  stem,   etc.    (pp.   141-152).  These 
"rules"  obviously  deal  with  matters  of  form  rather  than 
the  wording  ^f  content.     With  regard  to  the  wording  of 
essay  questions,   Coffman  advises  the   test-maker  to  make 
sure  that  essay  questions  are  "so  worded  that  all  candi- 
dates will  Interpret  In  the  same  way  the  task  to  be  done" 
(1971,   p.   287).     But  Coffman  offers  no  suggestions  for 
accomplishing  this  goal  other  than  to  put  the  essay 
questions  before  colleagues  for  review  or  put  them  on 
trial  In   the  field  In  a  manner  described  earlier  in  this 
report. 

"Field  tests"  and  " Item  analysis"  are  the  principal 
procedures  now  In  use   to  determine  whether  an  essay  test 
question  or  an  objective  test  Item  Is  well-constructed. 
Coffman  recommends  using  field  test  results  to  determine 
whether: 

1)  the  examinees  understand  the   Intent  of  the 
question,  or  whether  they  Interpret   it  in 
unintended  ways; 

2)  the  question  is  of  appropriate  difficulty 
for  the  examinees;  and 

3)  the  question  can  be  reliably  scored. 

In  Coffman's  view,   the   only  way  to  get  conclusive  answers 
to  these  questions  is   to  "obtain  a  representative  sample 
of  responses  and  conduct  an  experimental  scoring  session 
leading  to  the  generation  of  score  distributions  and 
estimates  of  reliability"   (p.   288).     Coffman  argues  that 
even  informal   testing  of  an  essay  question  with  a  small, 
unrepresentative  sample  of  students  is  likely   to  be 
better  than  no  pretesting  at  all,   for  "the  contrast 
between  the  type  of  response  anticipated  by  the  examiner 
and  the  type  actually  written  by  examinees ...  can  be 
dramatic"    (pp.  288-89). 

Interestingly,  "Item  analysis"  as  it  is  practiced 
in  psychome tr ics  does   not  take  into  account  the  linguistic 
properties     which  might  account  for   the  ambiguity  and 
difficulty  of   test  questions.     Rather   the  psychometric 
concept  of  Vltem  analysis"  refers  to  an  application  of 
statistical  techniques   to  determine  "difficulty"  and 
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"ambiguity."     The  "difficulty"  of  a  test  item  is 
"indicated  by   the  proportion  of  the  subjects  vho  get  the 
item  right"   (Wcod,   1960»  p.   81;  see  also»  Sax,  1974, 
pp.   229-254).     Item  "ambiguity"  from  the  point  of  view 
of  psychome tries  is  also  defined  quantitatively  accord- 
ing to  the  "extent  which  students  in  the  upper  group 
(top  quartile)   select  an  incorrect  option  with  about 
the  same  frequency  that   they  select  a  correct  one" 
(p.   233).     In  other  words,   "ambiguity,   is  defined  as  the 
inability  of  the  top  scoring  students   in  a  test   to  dis- 
criminate between  the  "correct"  and  the  "incorrect" 
alternatives  as  determined  by  theexaminer.     Sax  (1974) 
acknowledges  that  the  procedures  of  "item  analysis"  are 
such     that  "unfortunately   [they]   cannot  reveal  if 
ambiguity  results  from   the  students'   lack  of  knowledge  or 
from  poorly  written  items"   (p.   233).     It  is  important  to 
note  that   the  statistical  procedures  used  in  the  conven- 
tional field  test  of  essay  questions  enable  examiners  to 
determine  the  task's  difficulty  and  its  "power   to  discrim- 
inate" between  "successful"  and  "unsuccessful"  students, 
but  these  procedures  do  not  lead  to  an  analysis  of  the 
linguistic  properties  inside  the  texts  of  essay  questions 
which  might  affect  their  interpretation  by  the  persons 
responding  to  them. 

The  interaction  between  students  and  essay  test 
questions  is  recognized  and  demonstrated  by  psychometric 
research   (Coffman,   1971,  p.    289).     For  example,  one 
investigator   (French,   1966,   p.   588,  as  cited  in  Foley, 
1971,   p.    801)   identifies  "test  error"  among  the  sources 
of  error  in  scores  based  on  tests  requiring  actual 
writing:     "The  composition  test  is  almost  like  a  one-item 
test.     Some  students  may  happen  to  enjoy  the  topic,  while 
others  may  find  it  difficult  and  uns t imulating ;  this 
results  in  error."     Coffman   (1971)  goes  so  far  as  to  say 
that  "some  students  do  better  on  some  questions  while 
other  students  do  better  on  others.     To  some  extent  the 
grade  a  student  obtains  depends  on  which  quest  ions  appear 
on  examinations"   (p.   289;   see  also  Rosen,   1969).  Despite 
the  fact   that  researchers  in  educational  measurement  have 
acknowledged  questions-wording  effects,   the  larger  the- 
oretical issues  of  essay  task  construction  and  typology 
have  seldom  been  addressed.     There  has  been  little 
concern  about  determining  the  frequency ,  magnitude  or 
underlying  nature  of  essay  task-wording  effects.  Wording 
effects  h^ve  tended  to  be  treated  anecdotally  or  inci- 
dentally in  the  context  of  other  objectives  of  study.  For 
example,   the  fundamental  research  that  is  most  often  cited 
to  confirm  the  validity  of  the  use  of  multiple-choice 
questions  as  predictors  of  students*  writing  ability  is 
a  study  done  by  Godshalk,   Swineford,   and  Coffman 
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(1966),*     This  study  revealed   significant  variation  in 
ratings  assigned  to  the  essays  produce4  in  response  to 
the  five   topics  used  in  the  study,   as  reflected  in  the 
mean  score  for  topics.     The  investigators  point  out  that 
these  differences  in  ratings  would  have  consequences 
under  certain  examination  conditions: 

...if  the  five  topics  had  been  assigned  as 
alternate  topics  from  which  one  or   two  could 
be  chosen  by  students,  a  student's  rating 
might  depend" more  on  which  topic  he  chose  than 
on  how  well  he  wrote.     Or   if  one  topic  had  been 
assigned  to  one  form  of  a   test  and  another  topic 
to  a  second  form,   then  some  method  of  equating 
the   scores  would  be  required ;  o therwise  the 
magnitude  of  an  individual's  score  would  depend 
partly  on  which  form  of  the  test  he  wrote. 

Though  the  topic  effect  is  recognized  as  significant  in 
the  Godshalk  study,   it   is  considered  only  in  connection 
with  the  estimation  of   the  score  reliability.  Topic 
effect   is  identified  as  a  source  of  variability  in  the 
reliability  of   the  test  score,   but  it  is  not  discussed 
as  an  interesting  phenomenon  worthy  of  further  investi- 
gation. 

Professional  Wisdom  as  a  Source  of  Knowledge 

Practitioner  knowledge  develops  through  a  continuing 
involvement  in  the  unique  and  particular  experiences  of 
the  profession.     The  core  of  professional  knowledge 
derived   from  the  accumulation  of  insight  through 
systematic  self-inquiry  leads   to  the  fashioning  of 
various   degreesof  practical  wisdom.     This  practical 
wisdom  may  denote  a  high  level  of  competence  in 
practice,   but  it  remains  essentially  an  individual 
theory  of  practice  so  long  as  it  stays  private.  However, 
there  are  individuals  who  make  public  their  personally 
accumulated  wisdom  through  a   textbook  on  pedagogy,  a 
curriculum  report,   or  other  media.     Once  public,  these 
works  are  subject  to  critical  processes  which  may  lead 
to   their  verification,  validation,   and  refinement.  And, 
in  time,  we  have  the  development  of  a  form  of  theory  that 
might  be  called  "theory  as  idealized  practice."  ^ This 
kind  of   theory  contrasts  with   that  other   type  of  theory 
which  is  derived   from  systematic  inquiry  of  the  sort 
associated  with  formal  research  procedures. 

It  would  require  a  separate  research  project  to 
survey  and  systematically  collect  and  synthesize  available 

*    Martin  Steinman  has  cuallenged  the  conceptual  validity  of  this 
fatnous  study  (Research  in  Teaching  English,  I  (l)-.  Spring  1567, 
pp.  79-94). 
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theories  of  idealized  practice  covering  the  design  of  writing 
tasks.    Several  examples  from  different  periods  in  the  history  of 
teaching  writing  in  the  United  States  will  suffice  to  indicate  the 
type  of  normative  "rules"  recommended  on  the  basis  of  practice. 
Each  of  the  following  sets  of  Instructions  pertains  to  making 
assignments  for  classroom  instruction;  however »  they  seem  adaptable 
to  an  assessment  context  as  well. 

Fred  N.  Scott.    In  his  "Philosphy  of  the  Assignment,"  Fred 
N.  Scott  (1903)*  divides  the  making  of  an  assignment  into  five 
steps: 

1.  the  announcement  of  the  subject; 

2.  stimulation  of  interest  in  the  subject; 

3.  arousal  of  a  desire  to  write  upon  it; 

4.  suggestion  of  a  method  or  procedure  in  writing; 

5.  precautions  against  wasted  effort. 

Mi\ch  of  Scott's  discussion  is  devoted  to  consideration  of  the 
principles  for  selecting  an  interesting  subject.    He  considers  the 
feasibility  of  allowing  the  student  to  select  his  own  subject  and 
grants  that  this  is  a  valuable  mental  exercise.    But  in  practice, 
freedom  of  choice  does  not  work  because  too  much  of  the  effort 
expended  is  "mis-directed  and  desultory"  ,.(p*  321).    He  does, 
however,  recommend  carefully  prepared  special  exercises  in  subject* 
choosing.    Interestingly*  Scott*s  professional  Judgment  regarding 
subject  choice  anticipates  psychometric  research  data  which  indi- 
cates that  when  students  are  given  a  choice  of  topics »  they  are 
unable  to  relect  the  one  upon  which  they  will  make  the  highest 
score  (Coffman,  1971,  p.  291). 

Scott  offers  two  main  principles  for  choosing  subjects  for 
writing:     1)  the  subject  chosen  must  be  one  that  is  interesting 
to  the  teacher,  and  2)  the  subject  chosen  must  be  one  that  is 
interesting,  or  that  Qan  be  made  interesting*  to  the  students 
(p.  322).    The  first  principle  is  Important  because  it  reduces  the 
possiblity  of  topic  effect  upon  the  evaluator  when  the  principle 
is  observed.    As  Scott  explains. 

It  must  be  borne  in  mind  that  after  the  essays  are  written 
they  must  be  read,  and  it  is  the  teacher  who  will  read 
them.    He  ought  to  read  with  intense  interest  and  a 
kindling  enthusiasm.    But  if  the  subjects  are  distaste- 
ful to  him,  his  reading,  despite  the  most  conscientious 
efforts,  will  be  Hli|lf -hearted  and  ineffectual,  (p.  322). 

In  discussing  the  means  of  arousing  the  desire  to  write, 
Scott  claims  that  "such  a  desire,  when  it  is  natural,  springs 
from  two  healthy  impulses:    the  impulse  to  give  expression  to  ^ 
one's  thoughts  and  feelings,  and  the  impulse  to  communicate,  one's 
thoughts"  (p.  324).    Again,  Scott*s  professoinal  wisdom  anticipates 


*  I  am  indebted  to  Aired  H.  Grommoti,  Professor  Emeritus,  Stanford 
University,  for  calling  my  attention  to  this  source. 


ERIC 


6, 


A3 


the  arguments  of  latter-day  discourse  theorists  such  as  James 
Moffett  and  James  Brltton,  who  also  advocate  beginning  with 
assignments  that  elicit  personal,  expressive  writing  for  real 
audiences.    Scott  stresses  the  Importance  of  taking  account  of 
audience  In  making  writing  assignments:  "...In  assigning  the 
work,  the  teacher  should  be  at  some  pains  to  provide  an  audience 
or  a  reader.    Sometimes  the  audience  will  be  real»  sometimes  It 
will  be  Imaginary,  but  It  should  never  be  lacking"  (p.  325). 
Audience  Is  an  Important  consideration  because,  as  Scott  puts  It, 
"the  consciousness  of  waiting  auditors"  is  "the  most  powerful  of 
all  stimuli  to  expression"  (p.  325)* 

In  siiimnary,  Scott  states  the  goal  of  a  well-designed  writing 
assignment":  "The  effect  of  the  whole  should  be  to  create  a 
natural  situation,  real  or  Imaginary,  In  which  the  student's  powers 
of  expression  and  communication  are  stimulated  to  their  normal 
maximum"  (p.  326). 

Commission  on  English.    In  1965,  the  Commission  on  English, 
an  agency  of  the  College  Entrance  Examination  Board »  published  its 
report  on  the  teaching  of  English,  Freedom  and  Discipline  in 
English.    The  Commission's  report  contains  a  section  on  "Composi- 
tion  Assignments"  which  declares,  "No  part  of  an  English  teacher's 
job  is  more  important ...  than  the  making  of  sounds  well-framed 
assignments^  what  is  called  'providing  the  occasion'"  (p.  92). 
This  report  established  the  following  criteria  for  a  good  assign- 
ment (pp.  93-96): 

A  good  assignment 

1.  evokes  the  best  from  the  writer  and  gives  the 
teacher  the  best  chance  to  be  helpful; 

2.  aids  learning  and  requires  a  response  that  is 
the  product  of  discovery; 

3.  furnishes  data  to  start  from; 

4.  may  take  the  form  of,  or  be  construable  into^ 
a  proposition; 

5.  limits  either  form  or  content  or  both; 

6.  will  stipulate  the  audience  to  be  addressed, 
wherever  feasible. 

The  Commission  proposes  starting  an  assignment  with  data  such  as 
contradictory  criticism  of  a  literary  textt  opposing  arguments^ 
incongruous  bits  of  common  sense  (e.g.  contradictory  maxims).  The 
assignment  may  start  from  a  picture  or  a  cartoon^  but  often  it 
should  begin  with  the  literature  studied.    The  Commission  recom- 
mends converting  a  topic  into  a  proposition:  ^ 

"The  view  from  my  window"  has  the  virtue    of  inviting 
invention  but  the  serious  fault  of  giving  the  writer 
nothing  to  control  his  attention.    Translated  to 
propositional  form — "That  the  view  from  my  window  makes 


me  dread  (or  welcome)  getting  up  in  the  morning" — 
the  same  topic  suggests  a  focus  and  even  a  tone  for 
the  writer  to  exploit. . -The  provision  of  a  predicate 
immediately  puts  the  %nriter  into  a  posture  of  defense 
or  attack  and  calls  for  the  summoning  and  ordering  of 
evidence  or  arguments.    Instead  of  a  circle  circum- 
scribing undefined  matter,  the  proposition  supplies 
an  arrow  pointing  out  a  clear  direction  of  movement 
(p.  94). 

With  respect  to  consideration  of  an  audience,  the  Commission 
finds  that  "Too  many  English  themes  seem  addressed. .. to  the 
teacher. • .orp  in  a  vague  romantic  way»  to  the  world  or  posterity 
or  some  Saroyanesque  'You  Out  There'"  (p.  95).    The  Commission's 
remedy  for  this  lack  of  attention  to  audience  is  to  have  teachers 
vary  the  stipulations,  beginning  with  fairly  simple  exercises: 
"Wite  a  letter  to  the  governor  arguing  that...,'  'Write  a 
petition  to  the  student  council  ot  the  principal  requesting  that 
'Write  an  essay  on  the  proposition  that. ... 'etc."  (p.  95). 
The  Commission  expects  the  student  eventually  to  define  his  own 
audience  and  "to  define  it,  not  in  so  many  words,  but  by  tone  and 
content  alone"  (p.  96). 

The  Commission  does  acknowledge  that  "for  the  novice" 
especially,  it  may  be  inhibiting  rather  than  helpful  to  require 
that  he  think  of  a  specific  audience  for  a  poem  or  for  any  of 
the  range  of  forms  that  are  more  'literary'  than  expository  or 
argumentative"  (p.  95).     It  can  be  seen  at  once  that  the  Commis- 
sion's writing  assignment  models  actually  go  beyond  naming  an 
audience  to  specify  also  mode  or  genre  and  propositional  content. 
Thus,  these  assignments  circumscribe  and  define  the  essay  that  is 
to  be  written. 

Josephine  Miles.    Josephine  Miles  believes  that  the  solution 
to  dealing  with  subjects  of  any  magnitude  lies  with  training  the 
writer  to  formulate  a  "responsible  predication"  for  any  given  topic. 
In  her  view,  no  subject  is  too  large  or  too  small  or  too  complicated 
so  long  as  the  writer  controls  what  he  has  to  say  about  it.  This 
control  comes  through  the  selection  of  a  supportable  predication 
about  the  subject  from  the  writer's  own  experience  or  knowledge. 
Miles  suggests  that  one  can  take  even  an  abstract  subject  such  ad 
"death"  and  formulate  a  predication  that  is  "responsible"  and 
"supportable"  in  terms  of  what  a  writer  knows:    fot  example,  ''When 
death  appeared  in  my  life,  it  didn't  frighten  me"  (p.  3).  Miles 
also  points  to  the  structual  Implications  of  a  good  leading  sen- 
tence.   For  example,  the  development  of  the  main  idea  of  the  "death" 
predication  calls  for  a  retrospective  account  of  an  event  as  it 
happened.    The  temporal  locative  when  in  association  with  the 
sequence  of  verbs,  appeared  and  then  didn't  frighten,  together 
require  a  conmitment  to  develop  the  natural  temporal  sequence 
they  suggest.    Miles  believes  that  it  is  necessary  for  students  to 
learn  how  to  recognize  the  structure  that  is  implicit  in  their  ovm 
predications  and  in  the  questions  they  receive  in  assignments. 
Miles  has  formulated  her  theory  on  the  basis  of  years  of  experience 
in  teaching  freshman  English  and  through  informal  "experiments" 
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with  teaching  the  cues  to  the  structure  of  predications  in  state- 
ments and  questions*    She  says: 

Over  the  years •..we  have  come  to  agree  that  assignment 
by  topic  is  artificial,  not  only  because  it  is  unmoti- 
vated, but  because  it  is  too  far  removed  from  the 
central  unit  of  thought,  the  statement.    It  is  the 
question  to  be  answered,  rather  than  the  topic  to  be 
predicated,  which  gives  the  student  the  central  clue 
to  the  order  and  structure.    Only  after  he  can  handle 
questions  easily  can  he  move  easily  to  those  mere 
half-questions  which  are  topics  (p.  20). 

Josephine  Miles  criticizes  the  practice  of  offering  assign- 
ments for  writing  which  virtually  exclude  rational  processes. 
What  she  thinks  are  most  needed  are  opportunities  for  development 
of  essays  in  ideas  where  writers  can  make  "statements  based  on 
interest  and  speculation  and  [support]  them  by  adequate  evidence 
pro  and  con"  (p.  23).    The  power  of  students  to  compose  their 
thoughts  comes  through  essays  in  ideas  responsibly  developed. 

Commission  on  Writing.  In  1966,  the  Commission  on  Writing 
of  The  Education  Council  proposed  one  maxim  and  four  criteria  for 
preparing  composition  questions: 

Be  sure  that  any  question  you  ask  insures  the  possibility 
of  a  rhetorically  effective  composition. 

1.  A  good  question  should  be  stimulating:    it  should 
present  a  subject  matter  with  which  a  student  can 
readily  become  engaged  and  it  should  present  the 
subject  in  such  a  way  that  he  can  make  an  asser- 
tion about  it.    Not  "My  Summer  Vacation"  but 
"What  do  you  think  was  the  point  of  failure  in 
your  summer  vacation?" 

2.  A  good  question  should  be  fashioned  to  the  interest 
and  abilities  of  the  students.    Like  any  writer, 
the  poser  of  a  question  must  be  aware  of  his 
audience. 

3.  A  good  question  should  seek  to  elicit  a  specific 
response.    Although  one  may  at  times  wish  a 
student  to  generate  his  own  topic,  generally  one 
should  seek  to  show  the  student  the  specific  area 
in  which  he  is  to  organize  his  essay  and  the 
audience  for  whom  the  essay  is  intended.    A  teacher 
should  beware  of  his  own  power  and  not  force  the 
student  to  "write  to  the  teacher." 

4.  A  good  question  should  be  clear  and  precise  in  its 
instructions.    In  this  respect,  and  in  all  respects, 
the  demands  on  the  questioner  are  identical  with 
those  on  the  writer  (The  Education  Council, 


Commission  on  Writing  Competency,  1966,  p.  v) • 


William  F,  Irmscher,    In  a  manual  for  teaching  expository 
writing,  William  F.  Irmscher  (1979)  draws  upon  his  long  teaching 
experience  to  say  the  things  that  he  believes  teachers  engaged  in 
the  teaching  of  writing  should  consider.    He  offers  a  chapter  on 
"Topics"  which  he  opens  with  the  sentence:     "Assigning  topics  for 
writing  is  one  of  the  most  important  things  a  composition  teacher 
does"  (p.  69).    Irmscher *s  discussion  of  topics  echoes  in  many 
respects  the  principles  espoused  by  the  other  professionals  so  far 
presented  here.    Like  Scott,  Irmscher  would  not  give  certain  topics 
to  freshman  students  because    he    would  not  want  to  read  the 
cliched  essays  that  they  would  produce.     In  his  view  such  topics 
"are  not  springboards  for  writing;  they  are  traps.    They  don't 
give  students  a  chance  to  reveal  that  they  can  think  inventively" 
(p.  69).    Irmscher  offers  five  criteria  that  a  good  topic  ought 
to  meet  (pp.  68-72): 

1.  A  good  topic  ought  to  have  a  purpose. 

2.  A  good  topic  ought  to  be  meaningful  within 
the  student's  experience. 

3.  A  good  topic  ought  to  prefer  specific  and 
immediate  situations  to  abstract  and 
theoretical  ones. 

4.  A  good  topic  posing  a  hypothetical  situation 
should  be  within  a  student's  grasp, 

5.  A  good  topic  ought  to  encourage  a  student  to 
write. 

Irmscher  carefully  expl.  Ins  that  a  "meaningful"  topic  is  not 
necessarily  a  "personal"  topic,  though  it  should  be  involving. 
He  observes  that  students  do  distinguish  what  they  are  willing  to 
talk  about  in  the  dormitory  from  what  they  are  willing  to  talk  about 
In  the  classroom.    That  is,  students  consider  some  topics  "too 
personal"  (p.  70).    Irmscher  explains  also  what  he  means  by  speci- 
ficity in  a  topic  by  contrasting  the  abstract  topic,  "Discuss  Free- 
dom" with  another  one  that  is  more  immediate:    "List  the  freedoms 
you  enjoy  where  you  live  and  the  freedoms  you  are  denied.    What  is 
the  reason  for  the  denials?    Do  you  accept  the  reasons?    Write  an 
essay  on  the  subject"  (p.  70).    And  finally,  Irmscher  considers  the 
effects  of  topics  which  require  writers  to  proje'ut  themselves  into 
hypothetical  situations.    He  sees  a  topic  like  the  following  as  an 
imaginative  stimulus  which  implicitly  invites  satire:    "If  you  were 
the  first  chimpanzee  to  be  landed  on  Mars,  what  message  would  you 
deliver  to  Earthlings?"  (p.  71).    Hypothetical  topics  such  as  the 
following  one  provide  an  indirect  vehicle  for  expressing  thoughts 
that  otherwise  would  be  difficult  to  share:    "Write  an  extended 
obituary  notice  for  the  newspaper  upon  the  occasion  of  your  own 
death"  (p.  71).    This  kind  of  hypothetical  topic  actually  concerns 
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the  values  that  a  student  holds  now  even  though  it  seems  to  project 
him  or  her  into  the  future.    Thus,  Irmscher  finds  it  very  "different 
from  one  that  addresses  a  seventeen-year-old  by  saying,  'If  you 
were  an  old  man  or  woman,. This  latter  type  of  topic  is  ill- 
advised,  according  to  Irmscher:    "If  the  purpose  is  to  find  out 
what  it  is  like  to  be  old  or  how  the  old  think,  it  would  be  far 
preferable  to  have  young  people  talk  to  old  people  and  report 
their  impressions"  (p.  71).    Irmscher  also  believes  that  choice  is 
good  but  that  "limitless  choice  ('Write  on  anything  you  want  to') 
usually  proves  self-defeating.    Students  flounder  when  they  can 
choose  anything  in  the  world  (p.  72). 

Edmund    Farrell.    One  of  the  few  articles  to  appear  in  the 
English  Journal  (pp.  428-431)  exclusively  devoted  to  a  discussion 
of  the  making  of  composition  assignments  is  Edmund    Farrell' s  essay. 
Drawing  upon  his  experience  as  a  supervisor  of  student  teachers  in 
secondary  English  at  the  University  of  California,  Berkeley,  Farrell 
culled  a  number  of  writing  assignments  that  he  felt  exemplified 
various  sorts  of  lapses  in  making  assignments.    He  found  that: 

Common  to  these  assignments  is  the  absence  of  a 
stipulated  audience  and/or  purpose  which  would  help 
the  student  to  define  himself  in  context,  which  would 
lead  him  to  adopt  an  appropriate  persona  or  'speaking 
voice'  in  his  composition  (p.  430). 

Farrell  Joins  a  long  line  of  prof essoinals  who  advocate  specification 
of  audience,  role,  and  purpose  in  the  writing  assignment.  But 
Farrell  goes  further  than  most  other  writers  in  suggesting  patterns 
for  creating  assignments  with  built-in  "selves"  which  function  as 
cues  to  suggest  the  particular  roles  (and  "voices")  that  the  writer 
may  adopt  in  relation  to  audience  and  purpose.    Farrell  offers 
four  possiblilities: 

1.  The  "self"  of  an  assignment  may  be  internal  to 
the  selection,  the  t)arpose  unspecified,  and  the 
audience  private.     (Example:    Assume  that  you  are 
Lady  Macbeth  and  that  you  keep  a  diary.    Write  the 
five  entries  which  precede  your  suicide.)     (p.  430) 

2.  The  "self"  and  the  audience  may  be  internal  to  the 
selection  and  the  purpose  specified.  (Example: 
You  are  Tom  and  have  been  away  from  home  now  for 
three  months.    Write  to  Laura  trying  to  explain  to 
her  why  you  left.)     (p.  430) 

3.  The  "self,"  audience,  and  purpose  may  be  external 
to  the  selection.     (Example:    A  friend  of  yours 
comes  to  you  with  a  copy  of  Macbeth  and  says,  '*! 
understand  you've  read  this  play  in  class.  What 
should  I  look  for  so  1  can  most  fully  understand 
it?"    What  advice  would  you  offer?)  (p*  431) 
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4.    The  "self"  may  be  internal  to  the  selection  and  the 
audience  and  the  purpose  external.     (Example:  You 
are  one  of  the  inhabitants  of  Spoon  River  who 
has  died.    You  have  an  opportunity  to  speak  out 
from  the  grave,  summarizing  your  life  in  a  paragraph 
or  two.    What  comments  have  you  to  make?)  (p.  431) 

Farrell  believes  that  if  students  ate  given  good  assignments  that 
enable  them  to  adopt  appropriate  voices,  "their  writing  should  be 
more  pleasurable  to  read  and  much  easier  to  evaluate"  (p.  431). 

Arthur  J.  Carr.    A  professor  of  English  at  the  University 
of  Michigan,  Arthur  J,  Carr  made  a  kinescope  of  "Student  Writing 
Assignment  Based  on  'Fire  Walking  in  Ceylon/"  for  the  Commission 
on  English  in  the  1960's.    Carr  also  believes  that  the  assignment 
significantly  affects  the  quality  of  writing  the  students  may 
produce  in  response  to  it.    His  kinescope  production  relates  his 
experience    with  a  particular  writing  assignment  which  had  pro- 
duced a  terrible  set  of  papers.    But  instead  of  berating  his 
students  for  their  obtuseness,  he  takes  himself  to  task,  asking, 
"What  went  wrong  with  my  assignment?"    After  analyzing  the  things 
that  have  gone  wrong  in  making  the  assignment  (lack  of  preparation 
of  the  students,  lack  of  clarity  in  the  statement  of  the  assign- 
ment, lack  of  a  defined  purpose  for  making  the  assignment) >  Carr 
delineates  two  basic  assignment  patterns.    Broadly  speaking,  Carr 
sees  these  two  kinds  of  assignments  arising  from  a  reading  of  the 
essay  on  fire  walkers:    one  type  would  emphasize  structure  and 
form;  the  other,  ideas.    "If  we  restrict  the  choice  of  forms,  then 
we  should  liberate  ideas.    If  we  restrict  ideas,  then  we  should 
liberate  form."  Carr  believes  in  carefully  selecting  and  controlling 
the  number  of  constraints  a  student  must  cope  with  in  a  single 
assignment.    He  sees  the  making  of  assignments  as  something  to 
which  the  true  professional  gives  considerable  thought.    He  contrasts 
the  amateur  and  the  professional: 

...the  amateur  depends  on  inspiration,  and  this  means  that 
he  may  be  victimized  by  his  moods.    As  we  become  profes- 
sional, we  learn  to  carry  out  even  the  difficult  parts  of 
our  profession  expertly  and  regularly.    Unlike  the 
amateur,  the  professional  applies  principle  and  the 
res^ults  of  objectively  examined  experience  (p.  16,  1965). 

A  Professional  Consensus  on  Principles,    So  far  we  have 
considered  the  practical  wisdom  of  several  professionals  situated 
at  various  levels  of  instruction  with  regard  to  their  recommendations 
for  designing  writing  assignments  for  classroom  Instruction,    It  is 
not  known  whether  this  group  can  stand  as  representative,  for  no 
empirically  derived  sample  has  been    drawn       from  the  universe  of 
professionals  with  published  works  on  creating  classroom  inciting 
exercises.    Nevertheless,  to  add  more  cases  here  would  overwhelm. 
The  thirteen  members  of  the  Commission  on  English  and  seven  members 
of  The  Education  Council  plus  the  five  other  persons  presented  in 
this  section  together  do  constitute  a  noteworthy  professional 
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conimunlty  (see  Appendix  for  Institutional  affiliations).  Their 
collective  vlsdcm  seems  to  reach  consensus  on  several  principles  for 
designing  classroom  assignments  that  can  be  transferred  to  designing 
prompts  for  writing /assessments.    Consider  the  following  set  of 
principles. 

THE  PHILOSOPHY  OF  THE  ASSIGNMENT  FOR  WRITING  ASSESSMENT 

Maxim:    A  well*framed  assignment  for  a  writing  asses$m&i't 
provides  an  .CTccasion  for  writing  in  which  the 
student* 8  powers  of  expression  and  communication  are 
stimulated  to  their  normal  maximum. 

Principles: 

1.  The  subject  chosen  mu^  be  potentially  interesting 
to  the  teacher-reader  or  evaluator  of  the  essays 
written. 

2.  The  subject  chosen  must  be  potentially  interesting 
to  the  student  writers. 

3.  The  assignment  furnishes  data  to  start  from  which 
is  open  to  presentation  in  any  of  several  forms 
ranging  from  statements  or  questions  to  pictures 
or  cartoons  accompanied  by  instructions. 

4.  The  assignment  must  be  meaningful  within  the 
student's  experience. 

5.  The  assignment  seeks  to  elicit  a  specific 
response  and  limits  content  or  form  or  both. 

6.  The  assignment  is  itself  specific  in  suggesting 
an  audience  beyond  the  teacher  or  evaluator  when 
the  subject  makes  such  specification  feasible. 

7.  Assignment  by  topic  alone  is  artificial  and  yields 
a  lack  of  focus  in  development. 

Coda:    A  well-framed  assignment,  clear  and  precise  in  its 

instructions »  shows  awareness  of  the  audience  of  waiting 
examinees.    The  demands  On  the  assignment  creator  are 
identical  with  those  on  the  writer. 

Marjorie  Kirrie.    It  is  Interesting  to  contrast  this 
composite  set  of  principles  for  making  writing  assessment  prompts 
to  some  reconnendations  offered  by  a  professional  who  is  a  teacher 
and  a  chief  reader  for  the  College  Entrance  Examination  Board. 
In  a  brief  article  for  the  National  Writing  Project  Network  News- 
letter, Marjorie  Kirrie  (1979)  explains  why  it  is  necessary  to 
distinguish  between  the  instructional  and  the  assessment  situation 
in  conceiving  of  satisfactory  prompts.    Since  this  distinction  has 


already  been  discussed^Nh^fpre,  we  will  proceed  to  her  main  points. 


In  a  w6ll"-intentioned  effort  to  be  specific,  prompt  writers 
are  likely  to  unwittingly  introduce  factors  that  produce  unwanted 
side  effects.    These  side  effects  are  the  result  of  what  Kirrie 
calls  "overwriting"  the  prompt.     She  cautions  against  "shoring  up 
topics  with  all  kinds  of  directions:     'You  are  writing  to...,' 
•You  have  met  a...,'  'Tell  about...,'  'Tell  how...'"  (p.  7), 
Such  instructions  dictate  structure  and  lead  to  "a  stringing  together 
of  attempts  to  cope  with  each  'Tell'  in  the  exact  order  given  in  the 
prompt"  (p.  7).    Kirrie  also  cautions  against  asking  students  to 
address  a  specific  unreal  audience,  to  imagine  themselves  in 
situations  alien  to  their  world,  or  to  imagine  themselves  as  other 
entities.    The  prompt  that  begins  "If  you  were..."  can  only  be 
written  in  the  subjunctive  or  conditional  modes  and  sets  "traps 
for  all  except  the  highly  skilled"  (p.  7).    Unlike  the  other 
professionals  quoted  earlier,  Kirrie  maintains  that  specifying  an 
audience  makes  the  writing  task  needlessly  difficult  and  calls  for 
greater  skill  than  many  students  possess.    "Besides,"  says  Kirrie, 
"students  never  forget  the  real  audience  for  assessment  writing 
[teacher  and  other  educators]"  (p.  7).    Also,  Kirrie  notes  "some 
audiences  call  for  less  sophisticated  diction  and  syntax  than  we 
would  like  to  see  students  produce  in  a  test  situation"  (p.  7). 

So  what  does  a  prompt  author  need  to  do  in  order  to  make  a 
prompt  that  works?    Kirrie  advises  keeping  the  prompt  as  "non- 
directive  as  possible"  and  using  "any  word,  phrase,  or  brief  state- 
ment which  invites  a  variety  of  interpretations  and  response^*  (p.  7). 
She  says: 

A  statement  should  give  students  as  many  options  as  did 
the  quotation  from  Pogo  used  in  last  year's  College 
Entrance  Examination  Board  Test  essay  in  (L978>,  "We  have 
met  the  enemy  and  he  is  us,"  which  elicited  everything 
from  historical  exposition  to  commentaries  on  language 
usage.    As  for  additional  directions,  variants  of  "Write 
an  essay  on  what.. .means  (suggests)  to  you,"  together 
with  the  advice  that  examples  may  be  helpful  and  that 
good  writing  will  be  appreciated,  are  all  that  are 
needed  (p.  7). 

NAEP's  Primary  Trait  System.    While  Marjorie  Kirrie  represents 
the  professional  view  that  holds  that  a  writing  prompt  should  have 

the  widest  poss  ike  evocative  range"  (p.  7),  there  is  another 
view  that  is  diametrically  opposite  to  this  one.    The  National 
Assessment  of  Educational  Progress's  "primary  trait"  approach  calls 
for  a  precisely  defined  writing  "exercise."    Ina  Mullis  (1976)  of 
NAEP  has  described  the  characteristics  of  writing  exercises  devised 
for  assessments  using  a  primary  trait  system: 

The  essence  of  the  primary  trait  system  is  to  narrowly 
delineate  the  situation  of  the  writer,  by  defining  the 
variables.    With  this  approach  responses  should  all 
address  the  same  task  and  can  be  Judged  using  the  same 


f 


criteria.    Systema^tic  Judgments  provide  both 

descriptive  and  comparative  information  about  how 

well  individuals  ot  groups  of  people  can  use  writing 

to  coiamunicate  in  given  situations.    This  means  that 

for  each  exercise  three  things  must  be  specified: 

(1)  the  identity  of  the  writer  (whether  the  respondent 

is  himself  or  is  given  a  role  to  play),  (2)  the 

audience  (who  the  writer  is  writing  to)  and  (3)  the 

subject  matter  (what  the  writer  should  communicate 

to  the  audience) •  *  •  , 

The  more  structured  the  task,  the  less  difficult  the 
scoring,  since  the  essays  or  letters  will  be  more 
uniform  in  focus.    For  example,  if  persuasive  writing 
is  chosen  as  an  important  skill  to  measure,  it  could 
be  decided  that  students  should  be  able  to  use  writing 
to  influence  decision-makers.     The  taste  could  be, 
'Vritca  letter  to  someone  important  about  a  problem  in 
this  country."    A  better  task  would  be,  "Write  a  letter 
to  your  principal  suggesting  a  way  to  solve'  a  problem  in 
your  school."    If  the  nature  o£  the  problem  is  further 
defined,  e.g.,  the  lunchroom,  the 'task  would  be  even^ 
better.    With  a  national  sample, Jit  is  difficult  to  . 
identify  universally  applicable  situations.    For  a  . 
classroom,  school,  or  even  a  district,  the  problem  of 
respondents  having  common  knowXe^dge  and  experience 
should  be  alleviated  (pp.  8-9). 

Rex  Brown  (1978),  also  a  member  of  the  NAEP  staff,  presents 
some  ideas  about  how  to  develop  an  ideal  writing  assessment  Instry- 
ment.    He  also  speaks  on  behalf  of  the  stuctured  writing  task: 

If  you  want  to  evaluate  an  essay  for  certain  character- 
istics, then  you  must  be  sure  that  you  have  ^requested 
them  in  the  assignment.    This  is  not  a  trivial,  matter: 
it  is  extremely  difficult  to  write  assignments  that 
define  precisely  the  rhetorical  imperatives  that,  will 
either  be  met  or  missed  biy  the  students.    If  you  want 
to  kxiow  whether  they  can  elaborate  upon  a  role  expres- 
sively while  maintaining  control  of  a  point  of  view 
and  tense  then  you  have  to  set  the  task  up  in  such  a 
way  that  they  must  do  so,  and  define  acceptable  levels 
of  achievement  that  are  concrete  and  realistic  (p.  5). 

Discourse  Theory  as  a  Source  of  Knowledge 

Discourse  theory,  which  deals  with  the  development  of  full 
texts  in  speech  or  writing  (Klnneavy's  definition #  1971),  provides 
another  source  of  knowledge  that  influences  the  design  of  writing 
assignments  for  both  instructional  and  testing  occasions.  Discourse 
theory  with  roots  and  trad:^tions  reaching  back  to  antiquity 
attempts  to  describe  the  vaViet;ies  of  4i8course  and  the  elements 
which  contribute  to  the  making  of  texts,  spoken  6r  written. 
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Vatious  schools  of  thought  have  Mphasized  one  or  another  element 
such  as  the  writer,  the  subject,  or  the  audience,  either  Individually 
or  taken  together.    A  fourth  element,  function  (also  called  purpose 
or  aim)  arises  from  the  interaction  of  writer ,  subject,  and  audience. 
Related  to  ^11  of  these  is  the  concept  of  form  (also  called  mode  or 
genre) >    The  basic  problem  in  dealing  with  this  field  is  the  very 
richness  of  the  area  and  the  problems  attendant  upoa  selecting 
from  among  competing  theories.    Kinneavy  (1980)  sees  the  current 
condition  In  discourse  theory  and  rhetoric  as  comparable  to  the 
vigorously  competitive  scene  at  the  end  of  the  1950' s  when  various 
schools  of  linguistics  were  vying  for  dominance  (p.  37).    It  Is  out- 
aide  the  scope  of  this  review  to  treat  in  any  depth  even  a  very 
selective  sample  of  prominent  theories.    But  it  Is  important  ta 
consider  In  brief  outline  several  theories  which  often  provide 
organizing  principles  for  instruction  in  writing.    These  various 
discourse  schemes  Categorize  forms  and  functions  of  writing  that 
frequently  turn  up  as  specifications  and  constraints  in  writing 
prompts. 

Discourse  theory  has  gone  in  two  directions.  Traditionally, 
discourse  theorists  have  looked  at  the  products  of  writing  and 
sought  to  classify  what  they  found.    More  recently,  the  theorists 
have  attempted  to  describe  the  processes  of  writing  as  well  as 
the  works  created.    The  scope    of  discourse  theory,  including 
significant  traditional  and  emerging  contemporary* models,  is 
presented  in  Kinneavy  (1971)  and  D'Angelo  (1976)  although  neither 
of  these  authors  considers  the  work  of  James  Britton,  Janet  Emig, 
and  Linda  Flower  and  John  Hayes,  who  have  all  proposed  process 
models  of  discourse.  ^  ' 

Alexander  Bain.    D'Angelo  (1976,  p.  115)  credits  Alexander 
Bain  with  having  Established  in  1890  the  traditional  forms  of 
discourse  which  have  dominated  the  organization  of  writing  curricula 
and  testing  even  to  this  day.    The  traditional  forms  are  description, 
narration,  exposition,  and  argumentation,  although  Bain's  original 
set  also  included  a  fifth  form,  poetry.    According  to  D'Angelo' s 
account  of  Bain's  theory:  ^ 

Each  form  Is  assumed  to  have  its  own  function,  its 
own  subject  matter,  its  own  organizational  patterns, 
and  its  own  language  (p.  115).... 

...the  mind  can  be  divided  into  three  faculties:  the 
understanding,  the  will,  and  the  feelings.    The  aims 
of  discourse  (to  inform,  to  persuade,  and  to  please) 
correspond  to  these  three  faculties.    The  forms  of 
discourse  are  the  kinds  of  composition  that  relate  to 
the  faculties  of  the  mind,  the  aims  of  discourse,  and 
the  laws  of  thought.    Thus,  description,  narration, 
and  exposition  relate  to  the  faculty  of  understanding, 
persuasion  relates,  to  the\ill,  and  poetry  to  the 
feelings  (p.  116) . 


Bain*s  system  classifies  and  characterizes  written  products. 
Graphically  portrayed,  his  distinctions  are  as  follows  (based  on 
D'Angelo,  p.  115): 


Function 


Subject 


Description 


Narration 


Exposition 


Argument 


Evoke  acnse 
experience 


Tell  a  story, 
narrate  an 
•vent 

Inform , 
inatruct , 
preaent  ideas 

Convince , 
pcrauade, 
defend,  refute 


Objects  of 
aensea 


People  and 
event a 


Ideas , 

generalizations 


Issues 


Organization 


Space /time 


Language 


Space/time 


Logical  analysis 
and  classifies*- 

tion 

Deduction  and 
Induction 


Denotative  and 
connotative, 
figurative, 
literal, 

lapressiortlstic, 
objective 


As  above 


Denotative  and 
factual 


Factual  and 
MK>tive , 
depending  on 
appeal 


Leo  Rockas>    A  Contemporary  approach  to  the  classification 
of  written  products  is  offered  by  Leo  Rockas  (1964)  as  he  seeks  to 
combine  the  literary  modes  identified  by  Plato  and  Aristotle  with 
the  rhetorical  modes  identified  by  Bain.    Rockas  says: 

My  analysis  has  suggested  that  these  poetical  and 
rhetorical  modes  extend  themselves  naturally  and 
elegantly  to  the  concrete  modes »  description,  narra- 
tion, drama,  and  reverie;  and  to  the  abstract  or  plagal 
modes,  definition,  process,  dialogue,  and  persuasion. 
These  are  also  classified  by  means  of  procedure  as  the 
static  modes,  description  and  definition;  the  temporal  . 
modes,  narration  and  process;  the  mimetic  moieSf  drama 
and  dialogue;  and  the  mental  modes,  reverie  and  per- 
suasion.   Though  the  modes  may  be  modulated  and  mixed, 
1  consider  them  to  be  exclusive  of  each  other »  and 
together  inclusive  of  whatever  can  happen  in  discourse, 
at  least  at  the  simplest  level  of  rhetorical  analysis 
(p.  Ix.). 

Displayed  in  graphic  form,  Rockas'  reclassification  appears  below 
(pp.  xii-xv>*. 

1.    The  static  modes 

A.  Descriptions 

B.  Definition 
11.    The  temporal  modes 

A.  Narration 

B.  Process 
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III.    The  mimetic  modes 

A.  Drama 

B.  Dialogue 

IV.    The  mental  modes 

A.  Reverie 

B.  Persuasion 

The  A  categories  represent  the  concrete  modes  while  the  B  categories 
constitute  the  abstract  modes. 

James  Kinneavy.    Still  another  scheme  for  classifying  the  modes 
and  aims  of  discourse  was  proposed  by  Kinneavy  (1971)  in  a  compre- 
hensive* multi-disciplinary  examination  of  the  contributions  of 
classical  and  contemporary  rhetoric.    .Kinneavy  begins  with  the. 
communication  triangle  formulated  in  communication  theory  and  divide 
the  universe  of  language    into  three  fields  of  study  (syntactics, 
semantics,  and  pragmatics)  which  are  related  to  the  terms  of  the 
communication  triangle  (pp.  19-31).    The  communication  triangle  is 
depicted  below  (with  annotations  in  parentheses) : 


Encoder. 
(Writer) 
(Speaker) 


(Message) 
(Text)' 


rDecoder 
(Reader) 
(Audience) 


Reality 
(Subject) 

The  area  most  closely  tied  to  discourse  in  Kinneavy 'a  scheme  is 
pragmatics,  which  concerns  the  study  of  the  use  of  language  in 
actual  speech  (or  written)  situations  (p.  22).    According  to  - 
Kinneavy  the  aims  of  discourse  are  based  on  the  four  elements  of 
the  triangle.    When  the  focus  of  communication  is  primarily  on  the 
speaker-writer  and  the  aim  or  purpose  is  self-express^-^n,  the 
resulting  text  .is  expressive  discourse.    When  the  focus  is  on  the 
reader /Audience  and  the  speaker /writer's  aim  or  purpose  is  to  per- 
suade, the  resulting  text  Is  persuasive  discourse.    When  the  focus 
is  upon  the  subject  or  reality  under  consideration  and  the  speaker- 
writer  *s  aim  Is  to  explain  the  world  clearly  and  logically,  the 
result  is  referential  discourse.    When  the  focus  is  upon  the  text 
and  the  speaker-writer's  aim  or  purpose  Is  to  give  pleasure,  the 
result  is  literary  discourse.    To  summarize:    "Language  can  there- 
fore be  employed  with  the  stress  of  the  process  on  the  persons 
(encoder  or  decoder),  or  the  reality  to  which  reference  Is  made, 
or  on  the  product  (text  which  the  discourse  produces)"  (Kinneavy, 
p.  38). 

Kinneavy  sees  the  modes  of  discourse  as  "grounded  In  certain 
philosophic  concepts  of  the  nature  of  reality  considered  as  being 
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or  becoming"  (p.  36).    Thus,  the  modes  refer  to  classifications  of 
"kinds"  of  realities.    But  Kinneavy  alters  Bain's  traditional 
quartet,  keeping  description  and  narration  but  substituting 

evaluation  and  classification  for  argument  and  exposition.    Kinneavy  f 

rejects  exposition  and  argument  as  modes  because  he  considers  them 

aims  of  discourse  (1980,  p.  49).    In  other  words,  Kinneavy  uses  the 

term  "mode"  to  cover  "what"  is  being  talked  about,  and  "aim"  to 

cover  the  "why"  or  the  purpose  or  function  of  what  is  being  talked 

abo^t . 

Kinneavy  (1980)  summarizes  his  own  theory  of  discourse  and 
compares  it  to  the  theories  of  James  Moffett,  James  Br it ton,  and 
Frank  D'Angelo.    In  reaching  for  a  "pluralistic  synthesis"  of  four 
contemporary  models,  Kinneavy  identifies  a  number  of  compatibilities 
between  his  own  model  and  those  of  the  other  three  writers. 
Kinneavy  points  out  that  three  of  the  models  (Moffett,  Britton,  and 
Kinneavy)  are  "explicitly  based  on  the  semiotic  structure  of  the 
relationships  among  writer,  reader »  and  subject  matter"  (p.  38). 
He  then  goes  on  to  note  a  great  many  other  points  of  congruence 
aong  the  four  models,  but  he  does  acknowledge. that  "the  develop- 
mental dimension  found  both  in  Moffett  and  Britton  is  entirely 
lacking  in  Kinneavy  and  D'Angelo"  (p.  47).    Because  they  do 
address  the  developmental  dimension  of  composing,  both  Moffett' s 
and  Britton's  theories  will  be  presented  briefly  below. 

James  Moffett.    The  initial  presentation  of  James  Moffett 's 
discourse  model  appeared  in  Teaching  the  Universe  of  Discourse 
(1968).    Moffett  finds  his  discourse  model  in  a  Piagetian  theory 
of  Intellectual  development  which  conceptualizes  cognitive  develop- 
ment as  corresponding  with  the  growth  of  increasing  degrees  of 
abstraction.    Moffett  says: 

The  concept  that  I  believe  will  most  likely  permit 
us  to  think  at  once  about  mental  development  and 
the  structure  of  discourse  is  the  concept  of  abstrac- 
tion which  can  apply  equally  well  to  thought  and  to 
language  (p.  18). 

Thus,  Moffett  proposes  a  naturally  structured  theory  of  dis- 
course which  analyzes  the  subject  into  a  progression  of  speaker- 
audience  relationships,  subjects,  logical  sequence,  and  literary 
form.    His  most  recent  presentation  of  his  schema  appears  in 
Active  Voice;    A  Writing  Program  Across  the  Curriculum  (1981).  The 
main  lines  of  progression  are  as  follows: 
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Schema  of  Discourse 


(reading  down) 
Progression  of  Speaker-Audience  Relationship 


Thinking  to  oneself 

Speaking  to  another 
person  face  to  face 

Writing  to  a  known 
party 

Writing  to  a  mas8» 
anonymous  audience 


Inner  verbalization 
Outer  vocalization 

Informal  writing 

Publication 


Progression  of  Speaker-Subject  Relationship 


Recording  what  Is 
happening 

Reporting  what 
happened 

Generalizing 
what  happens 

Inferring  what 
will,  may»  or 
could  be  true 


Drama 

Narrative 
Exposition 


Logical 
Argumentation 


The  chronologic  of 
on-going  perceptual 
selection 

The  chronologic  of 
memory  selection 

The  analogic  of 
class  Inclusion 
and  exclusion 

The  tautologlc  of 
transformation  and 
combination 


On  one  dimension,  Moffett  displays  the  kinds  of  discourse  In  a  ' 
continuum  of  time  abstractions:     drama  records  what  Is  happening, 
xxarratlve  reports  what  happened,  exposition  generalizes  about  what 
happens,  and  argument  theorizes  about  what  may  happen.    The  temporal 
movement  Is  from  present  to  past  to  future.    On  a  second  dimension 
Moffett  orders  discourse  In  a  continuum  of  distance  (space)  between 
participants,  moving  In  abstractive  range  from  I-you  In  relation  to 
readers  and  from  I-it  In  relation  to  the  subject.    Moffett  symbolize 
this  latter  relationship  as  follows  (Moffett  and  Wagner,  1976, 
p.  15): 


7;; 
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SUBJECT  (it) 


The  audience  (reader)  relation  is  a  movement  from  self  through 
introspection  and  reverie  through  an  immediate  audience  to  a  mass 
audience  removed  in  time  and  space.    The  relation  to  subject  moves 
from  personal,  subjective  response  to  impersonal,  objective  analysi 

Writing  assignments  that  conform  to  Moffett's  theory  of  dis- 
course ask  the  student  to: 

.•.draw  subjects  from  actual  personal  observation  and 
to  abstract  this  material  in  ways  that  entail  increas- 
ingly sophisticated  and  artful  decisions;  assume  a  more 
and  more  remote  audience;  lead  from  vernacular  style  to 
literary  style,  from  improvisation  to  composition;  and 
open  up  for  the  student  progressively  higher  realms  of 
abstraction  (Moffett,  1981,  p.  6). 

James  Britton,    At  the  University  of  London  (1966-71)  James 
Britton  and  his  associates  have  developed  a  comprehensive  multi- 
dimensional model  of  discourse  derived  during  a  five-year  Schools 
Council  Project  on  the  Written  Language  of  11-18  Year-Olds.  This 
British  research  team  found  the  traditional  discourse  categories 
from  the  "predominantly  Scottish  tradition  of  rhetoric"  wanting  in 
a  number  of  respects: 

...they  are  derived  from  an  examination  of  the  finished 
products  of  professional  writers,  from  whose  work  come 
both  the  categories  and  the  rules  for  producing  instances 
of  them.    The  tradition  is  profoundly  prescriptive  and 
shows  little  inclination  to  observe  the  writing  process: 
its  concern  is  with  how  people  should  write  rather  than 
how  they  do.    It  can  scarcely,  therefore,  be  helpful 
in  studying  the  emergence  of  mature  writers  from  young 
writers  (Britton,  1975,  p.  4). 

So  instead  of  following  the  traditional  categories,  Britton' s 
research  team  developed  a  two-dimensional  model  which  on  one  plane 
classifies  writing  according  to  the  predominant  function  that  it 
performs.    On  another  plane  it  addresses  the  audience  dimension  in 
somewhat  the  same  manner  as  does  Moffett's  scheme. 
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Brltton*8  audience  scale  moves  from  the  self  to  the  teacher  to 
«n  unknown  audience.    An  innovative  elaboration  of  concept  of 
teacher  as  audience  appears  among  the  distinctions  portrayed  below 
on  the  scale  (pp.  116-129): 


SELF  OTHER 


TEACHER         WIDER  (KNOWN)        UNKNOWN  YOU 


Trusted    Gen' 1  Particular  Examiner 

Adult        Edu-  Educated 

cated  Prof es- 

Adult  sional 


The  main  categories  In  Britten's  scheme  of  the  functions  of 
written  utterances  are  shown  in  diagrammatic  fcrm  below  as  they 
are  presented  In  his  report  (p.  83): 


Mature  TRANSACTIONAL  EXPRESSIVE  POETIC 

Writer 


Learner  EXPRESSIVE 


The  expressive  form  of  writing  is  seen  as  "a  kind  of  matrix  from 
which  different         forms  of  mature  writing  are  developed"  (p.  83). 
The  expressive  form  is  defined  as  "language  close  to  the  self, 
revealing  the  speaker,  verbalizing  his  consciousness,  displaying 
his  close  relationship  with  the  reader."    Transactional  form  is 
defined  as  "language  to  get  things  done,  i.e.,  it  Is  concerned  with 
an  end  outside  itself."  The  poetic  form  is  defined  «s  "a  verbal 
construct,  patterned  verbalization  of  the  writer's  feeling  and 
ideas.      This  category  is  not  restricted  to  poems  but  would 
include  su^  writings  as  a  short  story,  a  play,  a  shaped  autobio- 
graphical episode."    Neither  the  expressive  nor  the  poetic  forms 
are  further  subdivided,  but  the  transactional  Is  divided  Into 
Informative  and  instructive  (conatlve) ,  and  each  of  these  has 
further  subdivisions.    The  Instructive  (conatlve)  is  divided  Into 
the  regulative  and  the  persuasive  functions.    The  informative 
is  divided  Into  categories  following  Moffett's  categories  of 
record,  report,  generalized  narrative  or  descriptive  information, 
analogic  with  a  low  level  of  generalization,  through  higher  levels 
of  generalization  and  speculative  (tautological)  forms  (pp.  88-105). 
The  informative  category  «t  the  lowest  level  records  concrete 
experience  and  moves  progressively  to  generalize,  abstract,  speculate 
and  theorize  at  the  highest  levels. 

Richard  Lloyd-^ Jones  and  Carl  Klaus.    Since  both  assessment 
and  instruction  are  based  on  explicit  or  assumed  models  of  the 
nature  and  purpose  of  discourse,  it  is  important  to  understand  the 
main  components  of  several  of  the  leading  models  that  now  dominate 


practice.    Thus,  we  have  examined  the  traditional  models  which 
classify  products  of  writing  and  the  leading  contemporary  models 
which  describe  processes  and  functions,  as  well  as/in  addition  to 
products  of  writing.    Lloyd-Jones  makes  a  similar  point  in  describing 
how  he  and  Carl  Klaus  selected  a  discourse  model  to  guide  the 
procedures  of  the  National  Assessment  of  Educational  Progress: 

In  order  to  report  precisely  how  people  manage  different 
types  of  discourse,  one  must  have  a  model  of  discourse 
which  permits  the  identification  of  limited  types  of 
discourse  and  the  creation  of  exercises  which  stimulate 
writing  in  the  appropriate  range  but  not  beyond  it  (p.  37). 

Consequently  Lloyd- Jones  and  Klaus  sleeted  a  three-part  model  after 
rejecting  several  others  as  being  either  too  simple  or  too  complicated. 
For  example,  the  model  b*»low  v^fi  rej#^cted  as  too  complicated  for  the 
purposes  of  the  national  assessment  (p.  40).    Even  so  it  remained  a 
useful  explanatory  model. 


Illocutionary 
(Spectator) 


Expressive 
(Encoder) 


Self-Oriented  ^ 


Literary 
(language) 
(code) 


Perlocutionary 
(Participant) 


High 
personal 
involvement 
in  situation 

A- 


Persuasive 
(Decoder) 


Other 
Oriented 


Explanatory 
(substance) 
(reference) 


Low 
personal 
involvement 
in  situation 


Although  they  acknowledge  that  usually  there  is  a  blend  of 
purposes  and  associated  forms  in  actual  discourse,  Lloyd-Jones  and 
Klaus  settled  on  a  tri-polar  scheme  from  which  they  expected  to 
derive  the  primary  rhetorical  traits  associated  with  each  type  of 
writing  represented.    The  model  they  finally  developed  follows: 
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Explanatory  Discourse 
(Subject  oriented) 


Expressive  Discourse 
(Dlscourser  oriented) 


Persuasive  Discourse 
(Audience  oriented) 


It  Is  this  simple  model  that  has  guided  the  design  of  exercises  for 
NAEP  since  1974. 

Skills  Model  Limitations,    The  discourse  models  of  Klnneavy, 
Moffett,  Brit^on,  and  Lloyd-Jones  and  Klaus  all  stand  in  contrast 
to  the  skills  model  of  writing  which  treats  the  subject  as  a  series 
of  discrete  activities  or  skills,  each  to  be  practiced  and  mastered 
In  turn.    Klnneavy  says: 

all  four  of  these  authors — Moffett,  Brltton,  D'Angelo 
[and  himself] — believe  that  composition  Is  best  taught 
with  examples  of  full  discourse.    The  concerns  with 
mechanics,  sentence  structure,  style,  even  Invention 
and  arrangement  are  best  seen  In  the  act  of  handling  a 
full  discourse.    Moffett  (1968,  pp.  205-206)  decries 
what  he  calls  the  particle  approach...   (1980,  p.  38). 

Klnneavy  continues  with  citations  from  each  of  the  other  aiithors  to 
show  their  preference  for  holistic  approaches  over  the  skills  model. 
It  should  be  evident  that  assumptions  about  the  nature  of  writing 
and  models  of  the  process  have  consequences  for  the  design  of  assess- 
ment Instruments. 

Although  the  discourse  theories  of  Klnneavy,  Moffett,  and  Brltton 
have  been  tested  In  classrooms,  none  except  Britton's  has  received 
extensive  trial  under  research  conditions.     It  is  time  now  to  look 
more  closely  at  the  research  contribution  to  the  design  of  prompts 
for  assessment  of  writing. 

Research  as  a  Source  of  Knowledge 

Basic  Research  Questions >    As  we  have  seen  there  are  a  good 
many  assumptions  in  practice  and  theory  about  requisite  features  in 
a  writing  instruction  assignment  or  assessment  prompt.    But  relatively 
few  details  are  yet  available  from  research  against  which  one  might 
check  these  assumptions  when  one  is  designing  prompts  for  writing 
assessments.    Various  researchers  in  the  field  of  writing  have  noted 
this  particular  gap  In  available  research  information  (Odell,  Cooper, 
and  Courts,  1978;  Odell,  1979;  Greenberg,  1981  Hoetker,    1981).  The 
kinds  of  questions  that  are  being  asked  indicate  the  current  areas 
of  need  in  the  study  of  the  means  used  to  elicit  writing  for  purposes 
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of  assessment: 


Questions  raised  by  Odell,  Cooper,  and  Courts  (1978,  p.  11) 

How  should  researchers  frame  a  %nriting  task  so  as  to 
obtain  the  best  possible  work  from  students? 

Must  researchers,  as  Sanders  and  Littlefi(^ld  (1973) 
claim,  provide  a  full  rhetorical  context,  that  is, 
information  about  speaker,  subject,  audience,  and 
purpose? 

Is  there  any  aspect  of  the  rhetorical  context  that  we 
need  not  include  in  a  %nriting  task?    Would  an  assign- 
ment that,  for  example,  specified  speaker,  subject, 
and  audience  but  not  purpose,  elicit  writing  that 
differed  significantly  from  writing  prompted  by  an 
assignment  that  specified  a  full  rhetorical  context? 

Questions  raised  by  Odell  (1979,  p,  41): 

Is  it  in  fact  true  that  different  kinds  of  writing 
tasks  elicit  different  kinds  of  writing  performance 
from  students? 

Are  there  some  kinds  of  tasks  in  which  purpose  seems 
a  more  important  consideration  than  it  does  in  other 
kinds  of  tasks? 

Does  one  vnriting  task  elicit  a  greater  number  of 
abstract  (or  connotative  or  formal)  word  choices 
than  do  other  tasks? 

Do  different  writing  tasks  lead  students  to  use, 
on  average,  longer  T-units  (or  more  final  free 
modifiers  or  more  adjective  modifiers,,.)  than  do 
other  tasks? 

Do  different  tasks  lead  students  to  use  different 
types  of  transitional  relationships  or  to  use  para- 
graphs that  fill  different  types  p^f  functions? 

Questions  raised  by  Greenberg  (1981,  p.  8): 

How  do  students  read  writing  tasks?    Which  aspects 
of  the  directions  do  they  understand?    Or  use? 
Or  ignore?  * 

How  do  students  interpret  %n:iting  tasks?    What  kinds 
of  details  do  they  thiijk  that  the  task  is  asking  for? 

How  do  students  react  to  witing  tasks?    What  fact6rs 
in  a  task  cause  a  student  to  perceive  it  as  "easy" 
or  "difficult?" 


How  are  students'  responses  to  writing  tasks  influenced 
by  situational  factors  (test  vs.  nontest  context,  in- 
class  vs.  out-of-class  assignment,  timed  vs.  untimed 
composition,  student-initiated  vs.  teacher-initiated 
tasks,  and  so  forth)? 

Literature  Search.    A  recent  search  of  the  literature  con- 
ducted at  the  beginning  of  this  study  in  fall  1981  confirms  that 
there  are  few  answers  to  the  kinds  of  questions  that  researchers 
are  asking  about  the  nature  of  the  effects  of  writing  prompts  upon 
performance.    For  this  study,  the  standard  data  bases  were  searched 
(ERIC,  Dissertation  Abstracts,  Psychology  Abstracts,  Social  Sciences 
Index,  Language  and  Language  Behavior  Abstracts  Index,  etc.). 
Using  such  descriptors  as  1)  writing  topic  effect,  2)  writing  topic 
analysis,  3)  writing  evaluation  criteria,  4)  writing  evaluation 
methods,  5)  writing  in  relation  to  assessment,  etc.,  some  ninety 
abstracts  were  thus  retrieved  and  followed  up.    But  this  search 
proved  disappointing  with  respect  to  the  number  of  items  dealing 
directly  with  properties  of  writing  tasks  and  their  effects  upon 
writers.    Only  one  item  in  this  set,  a  study  of  the  effects  of 
writing  for  different  audiences  in  three  different  modes  (Crowhurst, 
1978),  could  be  considered  to  reflect  a  research  interest  in  the 
differential  effects  of  writing  stimuli.    Nine  other  items  offered 
bits  of  information  about  writing  prompts,  embedded  in  the  context 
of  larger  research  interests  in  evaluating  writing. 

Inadequate  Research  Report  Practices.  Unfortunately, 
current  practice  in  reporting  research  results  tends  to  gloss  over 
or  ignore  topic  effects  in  the  very  studies  of  writing  where  topics 
h^e  initiated  the  written  products  under  investigation.  VJhen 
topic  effects  ^re  not  the  focus  of  experimental  interest  in  a  study, 
the  information  reported  about  the  topic — its  actual  text  and  the 
presentation  of  any  effects  noted — tends  to  be  cursory  and  oblique. 
Even  such  a  notable  and  widely  cited  study  as  the  Codshalk,  et  al 
(1966)  investigation  of  the  validity  of  the  multiple-choice  res- 
ponse mode  in  the  measurement  of  writing  neglected  to  quote  the 
actual  texts  of  the  five  criterion  exercises  used.    Godshalk  and  his 
associates  followed  the  rather  common  practice  of  describing  rather 
than  presenting  the  exact  wording  of  the  topic (s)  used  in  investi- 
gations of  writing.    This  reporting  practice  effectively  disallows 
a  reader  of  the  study  to  make  an  independent  judgment  about  the 
experimenter  *  8  classification  of  the  writing  tasks  and  to  make  an 
independent  appraisal  of  explicit  and  implicit  task  demands.  The 
consequences  of  this  practice  and  its  implications  for  retarding 
the  accumlation  of  data  about  topic  effects  are  examined  in  relation 
to  several  studies  where  topic  effects  were  noted  during  the  course 
of  an  investigation  into  other  factors. 

Nold  and  Freedman  (1977)  are  exceptional,  first  for  their  attempt 
to  control  for  topic  effects  in  their  carefully  planned  experimental 
study  of  factors  that  influence  readers'  judgments  of  essays,  and 
second,  for  'the  level  of  detail  which  they  devote  to  presenting 


their  essay  questions.     In  designing  their  study,  Nold  and  Freed- 
man  took  precautions  to  insure  that  their  results  would  not  be 
topic  specific  by  preparing  two  sets  of  two  questions  each,  parallel 
in  design  and  difficulty  and  requiring  writing  in  only  one  mode, 
(argumentative).     Since  Nold  and  Freedman  include  the  texts  of 
their  questions,  it  is  possible  to  inspect  them  and  make  independent 
judgments  about  their  claims  for  equivalence  in  design  and  difficulty. 
And  these  texts  do  become  a  matter  of  interest  when  they  report, 
"The  kind  of  essay  affected  the  amount  of  writing  produced...." 
The  two  matched  personal  opinion  topics  elicited  more  writing  and 
greater  variation  in  response  than  did  the  two  matched  quotation 
comparison  questions  which  elicited  less  writing  and  less  variation 
(p.  170).    Later  in  their  discussion  of  the  correlational  findings, 
Nold  and  Freedman  report  that  their  study  corroborates  earlier  ones 
that  found  length  to  be  a  predictor  of  quality:    "The  simple  correla- 
tion between  length  and  quality  in  our  sample  was  -.57,  indicating 
a  high  positive  correlation.    Our  study  shows  that  it  is  more 
damning  to  write  a  short  essay  than  elevating  to  write  a  long  one" 
(p.  173).    Although  this  study  does  present  considerable  evidence 
of  topic  effects  upon  essay  length  and  consequently  upon  its  quality 
rating  in  a  holistic  scoring,  this  particular  aspect  of  the  study  is 
never  directly  discussed  in  the  report. 

In  a  study  conducted  under  the  auspices  of  the  Center  for  the 
Study  of  Evaluation  at  the  University  of  California,  Los  Angeles, 
Winters  (1980)  set  out  to  compare  the  differential  effects  of  three 
types  of  scoring  systems  (impressionistic,  analytic,  and  frequency 
counts)  on  a  sample  collected  for  this  purpose.    The  report  indi- 
cates that  subjects  were  given  50  minutes  to  produce  a  200  word 
expository  essay  on  two  test  occasions,  one  week  apart.  "Two 
parallel  topics  were  randomly  assigned  to  subjects  and  counter- 
balanced to  control  for  test  occasion  (see  Appendix)"  (Winters,  1980, 
p.  7),    The  texts  of  the  writing  tasks  are  quoted  in  full  in  this 
appendix.    Winters 's  comments  in  her  discussion  of  the  findings  on 
topic  effects  take  up  only  two  sentences:    "Finally,  the  topics  used 
in  this  study,  which  appear  to  have  produced  some  unusual  effects 
on  writing  performance  were  expository.    There  is  no  evidence  that 
other  modes  of  discourse  or  even  other  genres  of  expository  topics 
would  have  produced  the  same  results"  (p.  19).    Although  this  condi- 
tion is  cited  as  a  limitation  upon  the  generalizability  of  the 
study's  findings,  there  is  no  explanation  of  what  the  "unusual  effects 
might  have  been.    One  must  turn  to  another  study  from  U.C.L.A.  which 
used  the  same  writing  tasks  for  illumination  of  the  "unusual  effects." 

Smith  (1980)  sought  to  compare  the  results  of  using  different 
measuring  strategies  in  assessing  high  school  students'  expository 
writing.    This  study  of  the  relative  merits  of  analytic  scales. 
Impressionistic  ratings,  and  objective  tests,  consisted  of  two  major 
components:     the  writing  topics  (and  directions)  and  two  forms  of 
rating  criteria.    According  to  Smith,  "The  topics,  designed  to  elicit 
like-sajDples  of  students  writing,  were  intended  to  promote  writing 
within  the  discourse  domain  of  exposition. . .Task  attributes  guiding 
development  of  the  topics  included  discourse  mode,  rhetorical  pur- 
pose, content  limits,  and  indended  audience"  (p.  6).    The  texts  of 
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Smith's  two  topics  are  provided  in  an  appendix^' Snd  are  identical  to 
the  two  used  by  Winters.     Because  Smith's  stifdy  found  a  pattern  of 
strong  relationships  across  analytic  subscales,  she  acknowledged  that 
it  would  be  "tempting  to  infer  that  the  Analytic  scale  actually 
tapped  a  single  unitary  dimension  of  writing"  (Smith,  1980,  p.  10). 
She  cautioned,  however,  against  making  such  an  interpretation 
because  of  "an  important  facet  of  the  writing  task  which  may  have 
affected. the  results — the  writing  topic  and  directions'*  (p.  10), 
Smith  suggests  the  possibility  that  the  "uniformity  in  the  rhetori- 
cal structures  of  the  majority  of  the  students'  writing  samples" 
may  have  resulted  from  the  structural  promptings  provided  in  the 
directions  accompanying  the  topics.     Ultimately,  Smith  raises  the 
question:     "Was  the  relative  lack  of  independence  among  the 
Analytic  subscales — and  the  high  correlation  with  Impressionistic 
scores — a  function  of  the  homogeniety  of  student  responses"  (p.  11)? 
An  independent  inspection  of  the  two    topics  suggests  the  likelihood 
that  there  could  well  have  been  the  interaction  between  topic,  ess^y, 
and  rating  as  indicated  in  Smith's  discussion. 

Another  study  from  U.C.L.A. 's  Center  for  the  Study  of  Evaluation 
is  concerned  with  the  validation  of  alternate  response  modes  for 
writing  assessment  (essays,  paragraphs,  and  "selected  response  foi>- 
mat").    Capell  and  Quellmalz  (1980)  sum  up  the  whole  writing  task  in 
Just  a  single  sentence:    "Both  narrative  and  expository  writing 
samples  were  elicited,  with  essays  being  on  the  topics  of  drugs  and 
violence,  and  paragraphs  on  the  topic  of  alcohol  use"  (p.  3).  And 
that  is  all  that  this  study  says  about  the  writing  stimuli  used  in 
this  experiment.    None  of  the  texts  of  the  instruments  used  in  this 
study  is  provided  in  the  report. 

One  does  not  have  to  read  very  many  research  reports  about 
writing  assessment  to  determine  that  there  is  no  uniformity  in  the 
practice  of  providing  the  texts  of  writing  tasks  and  other  relevant 
details.    The  importance  of  including  full  information  about  the 
writing  task  variable  catinot  be  overstated.    There  is  a  twofold 
loss  when  insuf f ir.ient  information  is  provided:     It  becomes  impos- 
sible to  make  an  independent  Judgment  about  the  claims  the  researcher 
makes  about  the  writing  task  characteristics,  and  it  also  becomes 
difficult  to  build  a  cumulative  knowledge  base  about  writing  task 
properties  and  related  effects.     For  example,  if  Kincaid  (1953) 
had  not  included  his  writing  assignments  In  his  study  of  factors 
affecting  the  quality  of  student  writing,  Braddock,  et  ail  (1963) 
could  not  have  later  reinterpreted  Kincaid 's  findings.  Whereas 
Kincaid  had  concluded  that  the  writing  performance  of  poor  writers 
varied  significantly  according  to  the  topic  assigned,  Braddock  and 
his  associates  read  the  results  differently.    They  noted  that 
actually  the  content  of  the  three  topics  was  very  similar,  but  they 
did  differ  in  the  modes  of  discourse  they  called  for.  Thus, 
Braddock  concluded  that  the  Kincaid  study  suggested  "that  variation 
of  the  assignment  from  expository  to  argumentative  mode  of  discourse 
did  not  seem  to  affect  the  average  quality  of  the  writing  of  a  group 
of  freshmen  who  were  better  writers  as  much  as  it  did  a  group  who 
were  worse  writers"  (p.  8).    On  this  basis,  Braddock  argued  that 
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until  more  could  be  known  about  the  effect  of  mode  on  writing 
performance,  it  seemed  necessary  to  control  for  this  element  in 
planning  assignments  to  be  used  for  research  on  writing. 

One  finds  frequent  reference  to  the  "classic"  studies  of  the 
reliability  of  the  grading  of  essay  tests  in  English,  mathematics, 
and  history  conducted  by  Starch  and.-'Elliott  (1912;  1913;  1913). 
For  example,  this  work  is  cited-*y  Coffman  (1971),  Foley  (1971), 
and  Sax  (197A),     Sax  even  quotes  Starch^  and  Elliott's  famous  ass^er- 
tion  about  the  unreliability  of  essay. test  ratings:     "It  is  almost 
shocking  to  a  mind  with  more  than  ordinary  exactness  to  find  that 
the  range  of  marks  given  by  different  teachers^  to  the  same  paper 
may  be  as  large  as  35  to  40  points"  (Sax,  p!  119).  Unfortunately, 
from  the  way  these  findings  are  often  cited,  one  might  easily  get 
the  impression  that  Starch  and  Elliott's  "paper"  was  an  essay  on 
a  single  topic.    Actually  the  two  "papers"  used  in  the  English 
part  of  this  study  were  not  essays.    They  had  been  Vritten  to 
answc^  a  set  of  six  discrete  questions,  among  which  were  the 
following:     "2)  Give  five  requirements  to  be  observed  in  the  struc- 
ture of  a  paragraph;  3)  Wr it e^^a  business  letter;  4)  Define  narra- 
tion, coherence,  unity;  classify  sentences  rhetorically  and  gifammati 
cally.     Illustrate  or  define.     5)  Name  all  the \masterpieces  studied' 
this  year  and  name  the  authot  of  each"  (from  Sta^h  and  Elliott  as 
reprinted  in  Payne  and  McMorris,  1967,  p.. 55).    What  is  truly 
shocking  is  that  such  a  crudely  conceived  study  is  still  being  cited 
as  a  credible  study  of  reliability  to  bolster  claims  about  the 
"disadvantages  and  limitations  of  essay  tests"  (cf.  Sax,  1974, 
p.  117).     If  the  actual  text  of  this  "essay  test"  had  been  omitted 
from  the  Starch  and  Elliott  report,  it  would  be  impossible  now  to 
see  how  misleading  their  research  findings  are.    Their  so-called 
essay  test  is  obviously  a  test  of  knowledge  of  discrete  items. 

/ 

One  can  only  wonder  about  the  nature  of  the  other  early  studies 
of  reliability  which  contributed  to  a  growing  lack  of  confidence  in- 
essay  examinations  during  the  1930' s  and  1940*8  and  that  ultimately 
led  to  their  virtual  abandonment  for  the  next  three  decades  in^favor 
of  the  more  reliably  scored  "objective"  test. 

Now  that  we  have  discussed  the  practical  problem  of  using 
research  as  a  source  of  knowledge  in  designing  prompts  for  assessing 
writing,  we  will  turn  to  consider  evidence  regarding  the  following 
elements:     1)  selection  of  the  subject  and  problems  of  prior  ^cnow- 
ledge,  2)  wording  of  the  question  and  the  problem  of  comprehension, 
3)  selection  of  mode,  4)  specification  of  rhetorical  context—role, 
audience,  purpose,  and  5)  contextual  influences.  , 

1 .     Selection  of  the  subject  and  problems  of  prior  knowledge. 
Various  teachers,  evaluators,  and  researchers  express  awareness  of 
the  need  for  judicious  selection  of  the  subject  or  topic  to  be  used 
in  an  assessment  or  research  experiment  (Lloyd-Jone's,  1577,  *p.  44; 
Harpin,  1976,  p.  92-93;  Hoetker,  1981,  p.  3;  Greenberg,  1981.. 
p-  3ff). 


William  Harpin  (1976,  p.  93),  in  reporting  the  results  of  a 


longitudinal  study  in  Junior  school  (ages  7.0  -  9.11)  involving  300 
students  and  50  teachers  in  nine  schools  in  England,  makes  the  point: 

A  theory  of  function  and  audience,  no  matter  how  elegant, 
is  incomplete  without  reference  to  the'  experience  that  is 
to  be  worked  on  and  aimed  at  a  reader.    Many  teachers 
and  certainly  the  majority  of  those  who  collaborated 
with  us  in  producing  the  writing,  see  choice  of  subject 
or  starting  point  as  the  most  formidable  challenge, 
dominating  all  other  concerns. 

Harpin'*5  appraisal  of  writing  assignments' is  based  upon  observa- 
tions during  the  normal  course  of  classroom  Instruction  in  the 
schools  participating  in  his  study.    Although  his  chapter',  "Con- 
texts for  Writing"  (pp.  91-110),  deals  with  writing  in  classroom 
instruction  contexts,  it  still  l^as  releyance  for  assessment  pur- 
poses. ,  Harpin  found  a.J^road  rangfe  of  "writing  situations"  and  he 
classified  22  kinds  of  experiences  used  to  S'timulate  writing. 
According  to^ Harpin,  the  nature  of  the^  stimulus  of feted  is  best 
described  as  "an  opp^sitipn  between  open  and  closed  or  convergent 
and  divergent  situations"  (p.  94).     He  ccJncludes  that  the  same 
experience  may  give,  rise  to  very  different  writing  outcomes, 
according  to  the  focus  of  attention  Suggested"  (p.  98).    A  single 
stimulus  may  be  presented  .a%  an  open  or  closed  prompt.  Harpin 
cites  an  example  wher^  two  teachers  in  different  project  schools 
and  working  independently  used  the  identical  stimulus,  a -picture 
of  fire-fighting  in  Jacobean  times,  as  a  starting-point  for  writing. 
Each  teacher  held  X  brief  discussion  prior  to  Jiavlng  the  students 
write.    One  teacher  asked  his  class  to  "write  a  story"  about  the 
picture  and  received  a  set  of  historical  fiction  papers,  all  except 
one  in  the  third  person.     During  the  pre-writing  discussion,  the 
other  teacher  considered  different  ways  the  picture  might  be  res- 
ponded to 'and  received  the  following  range  of  responses:     1)  the 
historical  fiction  form,  but  also  an  "eye-witness'*^  or  "participant" 
report  involving  a  change  from  third  to  first  person  narrative; 
2)  an  itemizing  of  the  content- of  thenpicture;  3)  a  broadly  ^esthetic 
.treatment  J^"all  th^  faces  lo0k  alike  ^nd  the  colors  are  very  dull")  ; 
and  4)  consideration  of, the  historical]  significance  (p.  94). 

HarpUn's' evalua/cion  of  photographs  or  pictures^as  stimuli  for 
writing^  is  instructive.    Apparently  equivalent  imagesf  ^'work  fit- 
fully or  not  at  all."    From  the  evide&e  of  experiments  and  ex- 
perience in  the  project,  Harpin  concluded  that  "one  significant 
feature  is  the  extent  to  which  interpretation  is  'closed*  or 
'open.'    There  Appears  to  be  a  necessary  level  of  ambiguity  for  a 
photograph  to  succeed  In  arousing  uniquely ^individual  associations. 
Too  much  ambiguity  and  nothing  happens;  too  iittle  and  the  responses 
show  little  or  no  variation"  (p,  106).    For  the  seven-  to  ten-year- 
olds,  human  Interest  seemed  "if  not  essential  then  desd^rable" 
(p.  106). 

Harpin' s  discussion  and  rf^assif ication  of  writing  tasks  (as 
/classroom  assignments)  (1976,  pp.  38-AA)  is  by  all  odds  the  most 
^  comprehensive  treatment  foulid  during  this  literature  review.  He 
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lists  the  following  possible  ways  of  classifying  writing  tasks: 
1)  by  content,  subject  matter  (what  it  is  about,  e.g.  the  Dewey 
Decimal  System  of  classification);  2)  by  form  (poetry,  prose,  essay, 
report,  etc.);  3)  by  audience  (learned,  popular,  the  distinctions  of 
rhetoric);  A)  by  writer-audience  relationstii^(Joos '  s  "five  clocks" 
with  its  account  of  the  interrelation  of  language,  style,  and  speech 
event  and  Moffett's  spectrum  of  distance  discussed  above);  5)  by 
writer  and  task  (creative,  free,  intensive,  imaginative,  practical, 
factual,  recording);  6)  by  function,  purpose,  intention  (expressive, 
poetic,  transactional  as  in  Britton  above;  or  to  tell  a  story,  to 
describe  a  thing,  to  produce  an  emotional  effect  as  In  Herbert  Read 
and  Bonamy  Dobree;  or  the  order  of  phenomena  in  spac^  in  descriptive 
writing,  the  order  of  phenomena  in  time,  in  historical  writing,  real, 
or  fictitious;  the  order  of  thoughts  in  the  mind  in  expository 
writing  as  in  H.J.C.  Grierson).    Ultimately,  Harpin  concludes  that 
an  almost  endless  sei.ies  of  classifications  is  possible:  "Writing 
will  refuse  to  fit  into  a  cut  and  dried  scheme  of  labels,  no  matter 
how  cunningly  or  patiently  constructed"  (p.  37).    What  is  important, 
Harpin  thinks,  is  to  select  the  scheme  that  is  of  greatest  value  in 
teachine,  and  we  might  add,  in  assessment.    Harpin's  work  has  been 
discussed  here  at  some  length  because  it  is  not  likely  to  be  readily 
accessible  in  the  United  States. 

The  discovery  of  fitting  subjects  for  assessment—ones  that  are 
neither  too  large,  too  insignificant,  nor  too  abstract— continues  to 
be  an  art  rather  than  a  science  because  there  appears  to  be  little 
more  than  anecdotal  information  available  about  which  subjects  "work 
and  which  do  not.     And  even  the  anecdotal  information  remains 
scattered  and  unanalyzed.     Characteristic  of  this  type  of  data  is  the 
National  Assessment  of  Educational  Progress  experience  with  certain 
subjects  as  reported  by  Lloyd-Jones  (1977,  p.  AA) ,  as  follows: 

Certain  images  create  trouble.    "Bananas,"  for  example, 
seems  to  provoke  pornographic  or  scatological  responses. 
Bill  collectors  seem  to  be  exceedingly  threatening  to 
some  people.    When  we  tried  an  explanatory-persuasive 
exercise  to  evoke  a  serious  letter  t^  correct  an  error 
in  computer  billing,  we  got  a  number  of  amusing  responses, 
but  when  we  tried  to  revise  the  Ctixercise  to  encourage 
humorous  literary-expressive  responses  by  making  the 
situation  iiiore  absurd,  we  found  that  the  increase  in 
the  number  who  could  Joke  about  computers  and  overdue 
bills  was  small  and,  strangely,  'the' number  which  dealt 
with  the  problem  in  serious,  highly  conventional  terms 
became  larger.    In  both  versions  a  substantial  number 
of  respondents  were  merely  hostile,   (emphasis  added) 

Lloyd-Jones  generalizes  that  expressive  exercises  tend  to  be  more  *^ 
enthusiastically  received  than  transactional  ones.     He  thinks  this 
is  the  case  because  expressive  writing  is  "to  some  extent  its  own 
reward.''    He  points  out  that  "Make-believe  transactions  present  a 
mild  contradiction  of  terms  and  that  may  tempt  one  to  underestimate 
the  skills  in  handling  practical  situations."    Lloyd-Jones  concludes, 
"...finding  likely  topics  within  the  range  of  all  the  respondents 


68 


and  challenging  enough  to  produce  the  desirous  efforts  despite  the 
lack  of  any  'payoff  remains  a  problem.". 

The  reports  of  three  studies  done  in  Florida  (Brossell,  1981; 
Hoetker,  1981;  and  Hoetker,  Brossell  and  Ash,  1981)  and  yet  to  be' 
published  offer  important  data  about  several  aspects  of  topics 
Including  the  effects  of  the  subject  choice  on  holistic  score. 
These  studies  were  conducted  by  Hoetker  and  Brossell  in  conjunction 
with  the  development  of  a  writing  examination  to  be  administered  as 
a  Teacher  Certification  Examination  after  1980.    Hoetker  (1979) 
completed  a  very  comprehensive  review  of  the  literature  for  the  * 
Florida  Department  of  Education  and  then  later  prepared  a  condensa- 
tion of  the  Earlier  work  which  is  referenced  here  (Hoetker,  1981). 
Brossell  (1981)  conducted  an  experimental  study  of  the  effects  of 
rhetoirical  specification  in  topics.    And  the  findings  of  both  of 
these  studies  "were  used  in  the  development  of  a  pool  of  topics  for 
future  administrations  of  t!*e  Teacher  Certification  Examination.  ^ 
This  work  in  Florida  and  the  work  of  Karen  Greenberg  of  Hunter  Col-^ 
lege  (1981a,  198^b)  are  among  a  hahdful  of  studies  in  which  the 
focus  of  interest  is  upon  the  topic  or  prompt  and  its  effect  upon 
writing  performance  ih  an  assessment  context. 

With  regard  ,to  subject  eff/cts,  Hoatiker  (1981)  identifies  a 
study  done  by  Rushton  and  Young  (1974)  wiiich,  Hoetker  reports, 

compared  essays  written  by  British  sixth-form  public 
school  boys  with  essays  written  by  factory  workers  of 
the  same  age.    On  academic  topics  the  language  from  the 
two  samples  displayed  the  expected  differences  between 
''elaborated"  and  "restricted"  linguistic  codes.  But 
when  the  two  groups  were  asked  to  write  on  a  technical 
subject,  the  linguistic  disadvantage  of  the  wrking 
class  boys  disappeared  (p.  6). 

The  Brossell  (1981)  study  of  rhetorical  specification  was  based 
on  six  topics  at  three  information  levels*^  (to  be  described  later  in 
this  report, under  the  section  on  rhetorical  specification)  that 
he  administered  under  simulated  test  conditions  to*  undergraduate 
majors  in  education  at  the  University  of  South  Florida  and  at  Florida 
State  University,    Brossell  found  that  mean  scores  or^.  the  six  sub- 
ject matters  (range  5.97  -  ^.35)  did  not  constitute  a  significant 
difference  (p.  11).    Although  the  statistical  analysis  did  not  reveal 
subject  effects,  Brossell  and  his  associate  Barbara  Ash  did  note 
"strikingly  superior"  writing  on  one  of  the  topics,  whereas  all  the 
others  elicited  essays  that  they  found  to  be  ^'a  desultory  lot."  On 
Topic  6  (level  2  version)  on  "violence  in  the  schools,"  the  writers 
addressed  the  topic  more  quickly,  had  more  to  say,  and  wrote  papers 
that  were  superior  as  a  group  and  received  the  highest  mean  score 
(7.05)  (pp.  12-13). 

Hoetker,  Brossell,  and  Ash  (1981)  field  tested  33  topics  with  a 
population  of  undergraduates  in  education  to  create  a  pool  of  vali- 
dated topics  for  future  administrations  of  the  Florida  Teacher  Certi- 
ficate Examination.    Their  procedures  enabled  them  to  determine  the 
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"popularity  of  topics  and  they  found  that  some  topics  were  clearly 
more  popular  (more  often  chosen)  than  others.     (The  topics  cannot 
be  revealed  because  of  tile  need  to  keep  the  pool  secure  for  future 
administrations  of  the  examination.)    But  these  researchers  report 
that  "except  in  two  extreme  cases,  popularity  was  quite  unrelated 
to  mean  scores  of  essays  written  on  the  topics"  (p.  10). 

^  In  addition,  the  students  in  this  study  had  completed  a 

questionnaire  which  asked  them  to  explain  why  they  had  rejected  the 
topic  on  which  they  did  not  write: 

•  ...the  student  comments  made  us  realize  that  we  had 
ignored  another  equally  important  dimension  of  the 
matter:     the  soihrce  of  the  essay's  content.    We  had 
not  taken  into  account  that,  even  if  all  students  are 
to  write  personal  essays,  it  makes  a  difference  whether 
the  topic  allows  students  to  give  their  opinions  on  a 
public  issue  or  demands  that  they  introspect  and  report 
their  inner  lives  (p.  10). 

The  researchers  found  that  they  could  classify  the  topics  into 
"pubric-"  and  "private"  categories  and  when  so  classified,  mean 
scores  were  almost  identical.    Because  student  preferences  are 
expressed  strongly  and  often,  they  recomnSfend  "that  student  prefer- 
ences be  taken  into  account  by  choosing  one  public  and  one  private' 
topic  as  optf^ons  for  each  form  of  the  examination"  (p.  11). 

Hoetker,  Brossell,  and  Ash ^did* find  three  bad  topics  in  the 
group  of  33  piloted.    These  three  topics — "American  neglect  of  the 
urban  environment,  "  "favorite  gadgets,"  and  an  invitation  to  des- 
cribe a  "dream  home" — were  seldom  chosen  and  produced  low  mean 
scores  (5.2).     Students  found  them  "difficult,  uninteresting, 
inappropriate,  and  requiring  special  knowledge  (p.  12),    The  re- 
'searchers'  comments  on  the  "dream  home"  topics  are  particularly 
illuminating:    one  of  the  researchers  had  had  excellent  results 
witfi  a  similar  topic  during  classroom  writing  instruction  when 
students  had  several  days  to  work  out  their  ideas.    But  on  the 
test  occasion  (written  in  the  45  minutes  allowed)  every  essay  was 
disorganized: 

Apparently,  the  students  had  fallen  into  daydreaming  on 
pape^*,  putting  down  each  new  idea  as  it  occurred  in  a 
stream-of-consciousness  fashion.    That  these  students  could 
not  write  well  on  this  topic,  which  had  been  successfully 
used  in  writing  classes,  is  one  reminder  that  testing 
writing  is  not  the  same  as  teaching  writing  (p*  13). 

Kajen  Greenberg  (1981b)  has  provided  information  about  nine 
questions  field-tested  at  the  City  University  of  New  York  for  the 
CUNY  Writing  Skills  Assessment  Test,  a  mandated  essay  test  given 
to  all  entering  freshmen  since  the  fall  of  1978  for  the  purpose  of 
diagnosing  writing  ability  and  proper  placement  in  writing  courses 
(Greenberg,  1981a,  p.  1).    Greenberg  provides  a  "popularity"  index 
for  these  topics  which  give  the  percentages  of  native  English  speak- 
ing and  ESL  students  who  liked  each  question.     At  the  time  of  this 
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report,  it  had  not  yet  been  established  whether  the  preferred  topics 
elicited  better  writing.    Both  populations  gave  the  following  topic 
the  highest  ranking  (83  percent  native  speakers,  81  percent  ESL) : 

It  is  the  responsibility  of  parents,  not  teachers,  to 
give  children  -information  about  sex.    Sex  education 
is  a  private  matter.     It  should  not  be  taught  in 
private  or  public  schools. 

Both  populations  gave  the  following  topic  the  lowest  rating  (38 
percent  native  speakers,  24  percent  ESL): 

The  government  is  considering  drafting  young  men 
into  the  army.    Young  , women  have  been  asking  for  equal 
rights  and  equal  responsibilities.     In  all  fairness, 
young  women  should  be  drafted  into  the  army  if  young 
men  are  drafted. 

Both  groups  give  the  same  rank  orders  to  all  the  other  topics  except 
one  on  which  there  is  a  substantial  difference  of  opinion.  Seventy- 
nine  percent  of. native  speakers  liked  the  following  question,  but 
only  52  percent  of  ESL  students  liked  it: 

Some  people  work  too  hard  and  too  much.     In  their  efforts 
to  succeed  at  school  and  at  work,  they  often  do  not  have 
time  to  build  good  relationships  with  their  families  and 
friends.    This  effort  to  succeed  is  not  worth  it. 

Additional  analyses  are  currently  being  made  of  the  results  of  those 
field-tested  prompts  used  in  the  CUNY  tests.    As  yet  they  are 
unavailable. 

Abraham  Stahl  (1977)  set  out  to  study  developmental  and  ethnic 
differences  in  the  writing  of  Jewish  youth  of  European  and  Oriental 
heritage  in  Israel  by  collecting  writing  samples  on  the  same  subject 
at  grades  2,  5,  8,  and  11.    The  topic  given  was  as  follows: 

My  home:    write  a  description  of  your  home,  its  rooms, 
and  their  contents  in  such  a  way  that  someone  who  has 
not  visited  it  can  form  an  idea  of  the  way  it  looks. 

Stahl  ran  into  unpredicped  subject  effects  , across  the  grade  levels 
assessed.    For  example,  many  of  the  students  of  European  heritage 
at  grade  ^.1  refused  to  cooperate  because  the  subject  seemed  "too 
childish"  to  them.  r 

It  is  easy  to  propose  a  number' 6f  maxims  about  the  selection  of 
subjects  for  assessment,  but  it  is  somewhat  more  difficult  to  be 
scientifically  exacting  in  selecting  the  subject  matter  that  we 
ask  our  students  to  give  their  heads  and  hearts  to  in  an  examination. 
We  know  that  we  must  somehow  find  subjects,  relevant  and  accessible, 
that  students  will  meet  with  enthusiasm.    But  just  now  researcl:^ 
provides  only  a  few  glimmers  of  how  this  goal  might  be  accomplished. 


2 .    The  Wording  of  Prompts  and  the  Problem  of  Comprehension, 
Braddock  (1976,  p.  120)  was  perhaps  the  first  to  question  the  dif- 
ference between  a  reading  test  and^ultiple-choice  test  of  written 
expression.    It  is  only  very  recently  that  the  multiple-choice  test 
instrument  itself  has  come  to  be  recognized  as  a  text  with  content 
and  linguistic  properties  that  presuppose  certain  reading  capacities 
whether  it  happens  to  be  called  a  mathematics  test,  a  writing  test, 
an  intelligence  test,  or  other  test.    All  of  these  tests,  as  well 
as  essay  tests,  are  first  of  all  tests  of  reading  comprehension* 
It  is,  of  course,  obvious  that  the  writing  task  in  an  essay  examina- 
tion must  be  understood  before  it  can  be  accomplished.  Both 
Hoifetker  (1981)  and  Greenberg  (1981b)  comment  on  the  reading  problem 
inherent  in  the  statement  of  the  writing  prompt.    Greenberg,  for 
example,  in  an  ex  post  facto  analysis^ of  the  results  of  a  study 
which  manipulated  topic  variables  "discovered  that  many  of  the 
students  who  participated  In  my  study  did  not  perceive  or  did  not 
understand  the  different  demands  of  the  experimental  questions" 
(1981b,  p.  8). 

There  appears       be  very  little  research  to  guide  test-writers 
in  the  wording  of  prompts  that  has  been  done  through  the  direct 
study  of  the  effects  of  different  wordings  of  prompts  upon  student 
writers  or  teacher  readers.    What  information  there  is  tends  to  be 
anecdotal  and  is  mentioned  in  passing  in  the  context  of  a  larger 
focus  of  research  interest.    For  exapple,  Harpin  comments  on  the 
significance  of  'wording  changes: 

where  intention  and  outcome  failed  to  match. . .teachers 
had  to  shoulder  much  of  the  responsibility.  Substituting 
one  word  for  another,  though  apparently  a  trivial  change, 
may  profoundly  affect  the  way  a  child  interprets  the 
task — "Describe  what  you  saw  as  you  walked  through 
the  fog"  gave  very  different  results  from  "Describe 
what  you  felt  as  you  walked  through  the  fog."  A 
similar  effect  is  caused  by  the  change  from  "What  does 
this  music  make  you  .think  of"  to  "What  does  this  music  , 
make  you  feel?"  (p.  109). 

Karen  Greenberg  (1982^^  comments  in  a  letter  on  the  problems  of 
devising  eight  topics  every  semester  in  order  to  test  about  30,000 
entering  students  at  QUNY.    She  says. 

The  opportunities  for  lexical  and  siyntactic  misinter- 
pretations of  test  topics  are  numerous.    For  example, 
last  semster,  one  of  the  topics  asked  aboiit  "vigilante 
groups  that  patrol  our  city's  streets  and  subways." 
(We  assumed  that  most  students  knew  about  groups  like 
the  'Guardian  Angels. •).. .we  got  responses  about  the 
'villain  groups,,'  the  'vigorous  groups,'  the  'Garden 
Angels, '  and  so  on. 

There  is  always  a  potential  problem  with  presumptions  about 
prior  knowledge.    The  "tennis  shoes"  topic  used  by  the  NAEP  and 
widely  studied  as  a  model  could  be  subject  to  regional  problems  of 
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comprehension.    James  D.  Atwater  (1981,  p.  13)  reminds  us  that  even 
such  a  seemingly  prosaic  term  as  "tennis  shoes"  might  have  to  be 
changed  to  "sneakers"  in  some  parts  of  the  country,  especially  in 
urban  areas.    Ruth  (1980,  p.  21A)  notes  that  "underlying  the 
assumed  common  language  of  questioner  and  respondent  there  may  be 
variable  coding  and  interpretation  procedures  that  must  be  taken 
into  account."    He  cites  an  example  given  as  an  illustration  of  in- 
adequate pre-testing  in  the  Michigan  Survey  Research  Center's 
Interviewer's  Manual  as  actually  an  example  of  variable  linguistic 
coding:     "In  one  SRC  economic  survey,  a  questioner  asked,  'Do  you 
think  the  government  should  control  profits  or  not?'  One 
respondent  replied,   'Certainly  not^    O^ly  Heaven  should  control 
prophets  (1969,  pp.  2-6)." 

The  whole  problem  of  "response  error,"  as  the  pheonomenon  is 
known  in  survey  research,  is  not  well-understood  in  writing  research. 
That  is  to  say,  there  is  no  theory  or  research  in  writing  assessment 
to  account  for  instrument  errors  which  derive  ftom  the  properties  of 
the  testing  instrument,  respondent  errors  which  derive  from  the  per- 
sonal characteristics  of  the  respondents — the  student  writers  and 
the  raters,  or  for  contextual  errors  which  arise  from  failures  to 
understand  the  special  conventions  of  testing  contexts  (e.g.  the 
request  to  write  a  personal  letter  to  display  competence  in  a 
testing  context  requires  a  different  performance  than  what  is  re- 
quired in  a  naturally  occuring  communicative  event) . 

In  an  effort  to  better  understand  the  effects  of  wording  essay 
questions  one  way  or  another,  Greenberg  (1981a)  set  up  an  experimental 
study  which  she  hoped  would  answer  questions  about  the  "agree/disagree" 
format  and  the  impersonal  nature  of  the  essay  question  format  used 
in  the  City  University  of  New  York  Writing  Skills  Assessment  Test 
administered  to  entering  freshmen  in  the  system. 

In  a  carefully  controlled  experiment,  Greenberg  (1981)  studied 
the  effects  of  varying  the  "cognitive"  and  "experiential"  demands  of 
essay  questions  on  the  writing  of  CUNY  entering  freshmen.  '  For  the 
purpose  of  the  study  she  designed  questions  with  "high  cognitive 
demands"  and  "low  cognitive  demands."  The  HCD  question  called  for 
an  evaluation  of  an  issue  and  involved  determination  and  application 
of  appropriate  criteria  of  evidence  as  well  as  a  capacity  to  dis- 
tinguish among  facts,  opinions,  and  values.    The  question  format 
called  for  the  writer  to  "agree  or  disagree"  and  defend  his  or  her 
position  with  logical  reasons.    The  LCD  question  called  for  an 
Interpretation  of  an  issue,  to  relate  issues  rather  than  to  Judge 
them,  and  it  provided  generalizations  and  criteria  for  making  judg- 
ments.   This  question  format  called  for  the  writer  to  "discuss" 
ideas  and  provided  a  number  of  options  for  structuring  the  response. 

Greenberg  also  sought  to  control  the  "experiential"  demands  of 
these  essay  questions,  and  thus  she  formulated  questions  with  "high 
experiential  demands"  and  "low  experiential  demands."    The  HED 
questions  invited  writers  to  relate  personal  experiences  in  response 
to  solicitations  cued  with  the  second  person  personal  pronoun  "you." 
The  LED  questions  encouraged  students  to  respond  in  a  less  personal 
way  with  factual  data  and  abstract  generalizations. 


The  questions  were  presented  in  uniform  forTnat3  consisting 
essentially  of  two  parts;  ^ 

1,  A  short  passage  (20-60  words)  which  introduced  the 
subject  to  be  written  uprfn. 

2.  The  instructions.    The  evaluative  questions  (high 
cognitive)  asked  the  question;    Do  you  agree  or 
disagree  with  this  statement?  ''And  they  also 
requested  supporting  explanatioji  and  illustration 
in  detail.    The  interpretive  question  (low 
cognitive)  asked  for  a  discussion  of  the  opening 
statement,  proposed  at  least  three  strategies  for 
accomplishing  the  task,  and  requested  reasons  for 
the  interpretations. 

Efforts  were  made  to  control  the  rhetorical,  propositional ,  and 
syntactic  difficulty  of  the  experimental  .questions  in  order  to 
determine  the  effects  of  the  two  main  variables,  the  cognitive  and 
experiential  demands.    Care  also  was  taken  in  the  selection  of  a 
subject  to  avoid  tasks  that  might  be  boring  or  threatening  or  out- 
side the  experience  of  the  writer.    Greenberg  expresses  her  awareness 
of  the  confounding  effects  of  controversial  subjects  such  as  "abor- 
tion'^ upot/both  writers  and  readers,  and  so  she  chose  "education" 
as  an  arei  of  shared  personal  experience  that  could  best  be  shaped 
into  the  Experimental  questions.     Sixteen  questions  were  then 
p-iloted  vJith  four  instructors  and  29  students,  and  eventually  eight 
questions  in  four  sets  of  two  each  were  determined  to  meet  the 
specifications  for  the  four  conditions:     1)  high  cognitive/low 
experiential  demands,  2)  high  cognitive/high  experiential 
demands,  3)  low  cognitive/low  experiential  demands,  and  A)  low 
cognitive /high  experiential  demands. 

Greenberg  hypothesized  that  there  would  be  a  significant  inter- 
action between  the  cognitive  and  experiential  demand  levels  of  the 
essay  questions  with  the  low  cognitive/high  experiential  demands 
eliciting  higher  overall  quality  of  writing,  as  measured  by  holistic 
scores.  Greenberg  expected  that  the  relative  eaS«  of  the  structured 
interpretive  question  (low  cognitive)  combined  with  the  interest  and 
accessibility  factors  of  personal  experience  subjects  (high  experi- 
ential) would  sustain  better  writing  performances  than  the  more  com- 
plex, more  remote  subjects. 

Greenberg 's  hypotheses  were  not  confirmed:     "None  of  the  four 
types  of  questions  elicited  a  substantially  superior  measure  of 
rhetorical,  syntactic,  or  lexical  performance"  (p.  58).    The  types 
of  changes  made  in  the  content  and  structure  of  the  questions  used 
in  this  experin^nt  did  not  have  any  statistically  significant  effect 
on  the  students'  quality  or  fluency  of  performance  as  measured  by 
holistic  score  or  as  determined  by  supplementary  analyses  such  as 
scorings  for  syntactic  complexity,  sentence  control  errors,  word 
form  errors,  and  essay  length.     Interestingly,  the  personalized  forms 
of  questions  which  used  the  second  person  pronoun  "you"  as  a  cue 
to  elicit  personal  experiences  did  not  produce  better  writing  or 
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even  get  a  higher  frequency  of  first  person  "I's"  in  response  (p.  59) 
Greenberg  speculates  that  this  latter  finding  may  have  resulted  from 
the  students'  belief  that  *ie    first  person  pronoun  is  inappropriate 
in  an  essay  produced  for  academic  purposes.    They  also  may  have 
little  faith  in  themselves  as  "reliable  authorities"  (p.  59). 
Greenberg  considers,  among  other  reasons  for  the  results,  the  possi- 
bility that  test  anxiety  may  have  constrained  students  from  compre- 
hending the  instructions  and  taking  full  advantage  of  suggested 
strategies.     She  also  thinks  that  the  fact  that  so  many  of  the 
students  in  the  sample  were  "inexperienced"  writers  who  were  in  or 
had  been  in  remedial  courses  may  have  affected  the  way  these  students 
attended  to  variations  in  question  wording  (p.  60). 

In  a  paper  related  to  this  study,  Greenberg  (1981b,  p.  8) 
comments, 

the  most  important  thing  I  learned  from  my  research 
is  that  the  relationship  between  any  writing  task  and 
students'  writing  performance  in  response  to  that 
task  is  profoundly  influenced  by  the  nature  of  the 
encompassing  situation.    The  effects  of  writing  task 
variables  differ  depending  on  the  nature  of  the  student 
and  the  nature  of  the  setting. 

In  interviews  with  some  of  the  students  participating  in  her  study, 
Greenberg  found  that  "many  had  not  even  read  the  entire  question — 
they  simply  read  half-way  through  and  then  began  writing  the  stan- 
dard five-paragraph  essay  that  their  English  teachers  had  always 
required  of  them..."  (p.  8).     It  would  seem  that  both  in  the  case 
of  the  Brossell  study  described  below  and  in  the  case  of  the  Green- 
berg study,  students  were  "reading"  the  context  of  the  school  test- 
ing situation  more  closely  than  they  were  reading  the  test  instru- 
ment itself.    They  may  have  thus  responded  automatically  with  what 
Lanham  (1979)  calls  "The  School  Style."    The  practical  wisdom  of 
meeting  the  demands  of  a  test  situation — writing  for  a  supposedly 
traditionally  oriented  evaluator-reader  and  avoiding  offense, 
grammatical  or  ideological — may  itself  beget  the  bland  strings"  of 
unqualified  assertions  that  so  many  observers  find  in  the  persuasive 
discourse  of  students  at  all  levels  (see  next  section  for  extended 
discussion) . 

Aside  from  the  Greenberg  study  just  presented  and  the  Brossell 
(1981)  study  to  be  considered  below,  the  larger  theoretical  issues 
of  prompt  design,  of  wording  and  information  load,  have  yet  to  be 
addressed  in  writing  research.    There  has  been  little  interest  in 
estimating  the  frequency,  magnitude,  or  underlying  nature  of  prompt- 
wording  effects.    For  this  reason,  it  is  difficult  to  find  prompt- 
wording  experiments,  and  we  have  only  scattered  anecdotal  information 
reported  as  illustrative  warnings,  but  not  further  developed  theoreti 
cally  or  empirically.    For  the  time  being  we  will  have  to  turn  to 
survey  research  (Payne,  1951;  Oppenheim,  1966;   Gordon,    1980)  and  to 
reading  assessment  research  which  has  recently  moved  In  this  direc- 
tion (Meier,  1973;  Kennedy,  1972;  Fillmore,  1981). 
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3.     Selection  of  Mode,    Of  the  several  variables  in  the  expres- 
sion of  writing  tasks,  one  that  has  received  considerable  research 
attention  ir,  the  specification  of  mode.    Since  Braddock  (1963)  called 
attention  to  the  neglect  of  control  of  the  variable,  mode    of  dis- 
course, by  people  doing  research  in  composition,  a  number  of  studies 
(Crowhurst,  1978;  Crowhurst  and  Piche,  1979;  Perron,  1977;  Whale 
and  Robinson,  1978;  Rosen,  1969)  have  been  conducted. 

In  a  series  of  studies,  Marion  Crowhurst  and  associates  have 
found  that  syntactic  complexity  is  affected  by  writing  task  variables 
such  as  the  specification  of  mode  and  audience  (Audience  will  be 
taken  up  later  in  this  review).    When  Crowhurst  and  Piche  (1979)  had 
sixth  and  tenth  eraders  write  in  the  modes  of  argument,  description, 
and  narration,  they  found  significant  differences  in  syntactic  com- 
plexity between  grades  6  and  10  in  the  modes  of  argument  and  des- 
cription but  not  in  narration.    They  also  found  that  the  difference 
in  writing  Xask  produced  a  greater  difference  between  students  in 
the  same  grade  than  was  evident  in  the  comparison  of  the  work  of 
students  four  grade  levels  apart.    That  is,  there  was  a  greater 
difference  in  T-unit  length  between  narratives  and  arguments  at 
grade  10  than  there  was  across  all  three  modes  in  grades  6  and  10, 

In  another  study  (1978)  Crowhurst  found  argument  papers  more 
syntactically  complex  than  narrative  papers  at  grades  6,  10,  and  12. 
Since  her  findings  are  in  agreement  with  Perron,  1976;  Rosen,  1969; 
San  Jose,  1972,  Crowhurst  (1978)  concludes: 

It  may  be  regarded  as  well  established  by  research 
that  a)  differences  in  syntactic  complexity  are 
associated  with  differences  in  mode  of  discourse, 
and  b)  narrative  writing  is  generally  less  syntacti- 
cally coiiiplex  than  argument  (p.  8). 

Harold  Rosen's  findings  are  in  a  Ph.D.  thesis  (1969)  which, 
though  not  published,   had  been  made  available  with  the  author's 
permission  in  summary  form  by  John  Pearce  (1974)  in  a  Schools  Council 
Publication.    Rosen  sampled  fifth  year  pupils  in  a  variety  of  schools 
on  the  0-level  Examination  pattern  (Ordinary  Level  in  English 
Language  in  the  General  Certificate  of  Education)  in  response  to  a 
series  of  writing  tasks.     He  found  a  clear  difference  in  the  diffi- 
culty of  various  types  of  writing.    "Narrative/descriptive"  writing 
is  least  difficult,  "discussion/speculative"  writing  is  the  most 
difficult.    Also  he  found  that  differences  between  an  Individual 
student's  performance  on  different  kinds  of  writing  could  be  very 
great.    The  most  satisfactory  measures  of  linguistic  maturity  appear 
to  be  of  limited  validity  and  correlate  more  strongly  with  the  kind 
of  writing  being  attempted  than  with  the  overall  competence  of  the 
writer.    Fifth  year  pupils  do  not  progress  in  writing  during  the 
year.    As  their  perception  of  the  "ideal"  examination  essay  develops, 
their  spontaneity  and  competence  remain  static  or  decline.  One-word 
essay  titles,  instead  of  allowing  for  flexible.  Imaginative  treatment, 
yield  uniformly  less  good  results  than  others.    Rosen  found  the  best 
writing  was  generated  by  open-ended  literary  or  poetic  stimuli  which 
encouraged  commitment  without  requiring  self-exposure.    Rosen  found 


that  students  in  the  fifth  form  were  not  yet  prepared  to  meet  the 
examiners'  model  of  "good  writing,"  discursive  prose  which  adopts 
a  detached,  speculative  treatment  of  a  general  or  public  issue. 
Rosen  concluded  that  given  the  unknowns  in  the  formulation  of  essay 
topics,  the  strongest  possible  reservations  were  in  order  with 
regard  to  the  matter  of  single-sample  testing  and  the  absence  of 
pre-testing  (pp.  55-56). 

In  his  review  of  the  literature  on  topic  effects,  Hoetker  (1981) 
reports  on  studies  by  Rosen  and  others  which  have  found  similar  con- 
nections between  mode  of  discourse  and  syntactic  complexity.  In 
addition  to  these;  there  are  still  several  other  sources  that  focus 
on  the  difficulty  students  have  with  writing  argument  or  persuasive 
discourse  (Cooper,  1979;  Neilson,  1979;  Miller,  1980),    There  seems 
to  be  no  doubt  that  writing  argument  is  difficult  for  students  at 
all  levels  where  it  is  assigned  through  early  college  levels.  But 
Freedman  and  Pr ingle  (1981)  are  the  first  to  shed  some  light  on 
possible  causes  of  the  difficulty.    In  a  study  of  the  capability 
of  12  and  13  year-old  students  in  composing  arguments  as  opposed  to 
narratives,  they  founcj  that  these  students  were  far  more  successful 
in  realizing  the  conventional  schema  for  story  structure  than  for 
the  structure  of  an  argument  (98  percent  of  the  students  could 
embody  narrative  structure  in  their  stories  but  only  12.5  percent 
could  realize  the  classical  argument  patterns),    Freedman  and 
Pr ingle,  in  discussing  their  findings,  feel  it  is  too  simplistic  to 
attribute  this  failure  to  create  well-wrought  arguments  to  "lack 
of  organization"  or  "lack  of  a  sense  of  form  or  structure"  because 
these  students  demonstrate  extraordinary  deftness  in  organizing 
narrative  material.    Thus,  they  conclude  that  while  students  have 
multiple  opportunities  to  internalize  narrative  schemas,  nd  such 
parallel  opportunity  exists  to  inculcate  argument  patterns. 
Freedman  and  Pringle  also  discuss  the  developmental  aspect  of 
written  argument  and  its  requirement  of  cognitive  maturation.  To 
be  able  to  write  argument,  students  must  have  had  sufficient 
exposure  to  the  genre  through  reading  and  time  to  allow  their 
maturation  levels  to  enable  them  to  cope  with  complexities  of-  the 
form. 

These  findings  about  differences  in  the  difficulty  of  modes 
would  seem  to  have  implications  for  the  selecting  of  subjects  arid 
the  preparation  of  accompanying  instructions.    Just  how  phases  of 
cognitive,  affective,  and  moral  development  need  to  be  taken  into 
account  in  the  formulation  of  writing  assessment  tasks  is  dealt  with 
at  great  length  in  Wilkinson's  (1980)  account  of  the  work  of  the 
Crediton  Project  in  Crediton,  Devoi    (U.K.),  1978-80.  Wilkinson 
and  associates  studied  the  development  of  the  written  language  of 
children  7-14  on  four  planes—cognitive,  affective,  moral,,  and 
stylistic.    Four  compositions—narrative,  autobiographical, explan- 
atory, and  argumentative— were  requested  from  groups  of  children 
at  ages  7,  10,  and  13  in  the  context  of  their  normal  lessons.  , 
The  same  four  subjects  were  given  to  each  group  and  the  results  were 
comparj^d  according  to  four  "models"  for  the  analysis  of  writing — 
cognitive,  affective,  moral,  and  stylistic.    The  study  attempts  to 
demonstrate  the  need  for  establishing  criteria  to  Judge  composition 
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in  a  developmental  context.    No  simple  sutnmary  of  this  work  Is 
possible,  for  the  authors -^rgue  that  qualities  of  thought*  feeling* 
moral  judgment  and  style  all  must  enter  into  a  process  of  continuing 
assessment  conducted  as  an  integral  part  of  the  day-to-day  instruc- 
tion. 

4.     Specification  of  Rhetorical  Context — Role,  Audience, 
Purpose.    Of  all  principles  espoused  for  designing  the  writing 
assignment,  specification  of  a  full  rhetorical  context  is  the  one 
that  seems  to  be  most  strongly  established  in  the  canons  of  practice. 
The  lineage  of  this  notion  goes  back  to  classical  rhetoric,  and  it 
has  found  vivid  restatement  at  various  times  in  tlie  history  of  writing 
instruction.    P.B.  Ballard  (1939)  gives  an  account  of  it  in  his 
chapter  on  "The  Hartog  Method,"  based  upon  the  work  of  Sir  Philip 
Hartog  as  it  appeared  in  The  Writing  of  English  (1907).  Ballard 
sets  forth  the  first  principle  in  Hartog' s  method: 

7 

...a  pupil  should  never  be  required  to  write  a  composition 
without  his  having  clearly  in  mind  the  audience  for  whom 
it  is  intended,  and  the  object  he  has  in  view  in  writing 
it.    This  principle  is  flagrantly  violated  not  only  in 
school  exercises  but  also  in  the  essays  set  at  public 
examinations.     Current  practice  is  neatly  summed  up  by 
Sir  Philip  Hartog  in  the  formula:     "Write  anything  about 
something  for  anybody."    Nobody  does  this  in  ordinary 
life.     To  quote  Samuel  Butler:     "It  takes  two  people  to 
say  a  thing — a  sayee  as  well  as  a  say^er.     The  one  is  as 
essential  to  any  true  saying  as  the /Other."    He  might 
have  added  that  something  else  is  ^^sentlal:    a  motive 
for  saying  it.... Nobody  writes  for'  the  mere  fun  of 
expressing  himself  on  paper:    he  aiWresses  himself  to 
somebody  whom  he  wants  to  influence;  he  wants  to  question 
him,  to  inform  him,  to  persuade  him,  or  to  convince  him. 
He  has  always  at  the  back  of  his  mind  this  possible 
reader  or  circle  of  readers — a  friend,  a  coterie,  or 
the  world  at  large.    Without  this  recipient  and  respon-  . 
sive  factor  the  business  of  writing  resembles  an  attempt 
to  work  a  battery  with  only  one  terminal.    Whatever  else 
you  may  get,  you  will  get  no  electricity  (pp.  79-80). 

Many  professionals  have  recommended  full  specification  of  the 
rhetorical  context,  perhaps  less  eloquently  but  none  the  less 
emphatically  than  Ballard.    Thus  it  is  not  surprising  that  Hoetker 
(1981)  was  convinced  at  the  end  of  his  original  review  of  the  lit- 
erature that  full  rhetorical  specification  was  desirable.  Hence, 
he  recommended  in  that  original  version  that  topics  for  Florida's 
Teacher  Certification  Examination  "should  take  the  form  of  scenarios 
simulating  writing  tasks  likely  to  come  up  in  the  course  of  a 
teacher's  work"  (p.  11).     But  the  results  of  Brossell's  (1981)  study 
of  the  "information  load"  of  topics  caused  Hoetker  to  change  his 
recommendation  to  favor  briefer,  less  fully  specified  topics  instead. 

In  1980.  Hoetker' s  colleague  at  Florida  State  University,  Gordon 
Brossell,  ran  an  experiment  to  test  the  hypothesis  that  essay 
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questions  that  specify  a  full  rhetorical  context  are  superior  to 
less  complete  versions  of  the  same  questions.    Brossell  used  six 
topics  written  on  three  information  levels  to  generate  writing 
samples  to  evaluate,  by  holistic  scoring  procedures,  the  effects 
of  information  level  and  subject  (p.  3): 

Level  1,  low  information  load,  presented  the  topic  in 
a  brief  phrase.    Decisions  about  audience, 
purpose,  form,  mode  of  discourse,  voice  were 
left  to  the  writer  with  no  guidance  provided. 

Level  2,  moderate  information  load  presented  a  topic 
containing  a  general  introductory  statement. 
It  then  posed  a  what  or  a  why  question  asking 
for  expression  of  personal  opinion. 

Level  3f  high  information  load,  presented  a  hypothetical 
situation  and  requested  the  expression  of 
personal  opinion  as  in  level  2.     But  this 
version  also  specified  role,  audience, 
purpose  and  form  or  mode. 

Topic  6  is  quoted  here  in  its  three  levels  of  specification. 

Level  1:     Violence  in  the  schools. 

Level  2:     According  to  recent  reports  in  the  news  media, 
there  has  been  a  marked  increase  in  incidents 
of  violence  in  public  schools.    Why,  in  your 
view,  does  such  violence  occur? 

Level  3:     You  are  a  member  of  a  local  school  council 
made  up  of  teachers  and  citizens.    A  recent 
increase  in  incidents  of  violence  in  the 
schools  has  gotten  widespread  coverage  in  the 
local  media.    As  a  teacher,  you  are  aware  of 
the  problem,  though  you  have  not  been  personally 
involved  in  an  incident.    At  its  next  meeting, 
the  council  elects  to  take  some  action.  It 
asks  each  member  to  draft  a  statement  setting 
forth  his  or  her  view  on  why  such  violence 
occurs.     The  statements  will  be  published  in  the 
local  newspaper. 

Write  that  statement  expressing  your  own 
personal  views  on  the  causes  of  violence  in 
the  schools. 

The  other  five  topics  dealt  with  1)  the  ''basics"  of  education, 
2)  teaching  as  a  career,  3)  discipline  in  the  classroom,  4)  the 
four-day  work  weeV,  and  5)  American  wastefulness. 

The  six  copies  were  randomized  and  administered  to  a  sample  of 
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360  undergraduate  education  majors  at  Florida  State  University  and  at 
the  University  of  South  Florida,    A  panel  of  three  raters  read  and 
holistically  rated  the  essays  according  to  criteria  developed  for  the 
examination.    The  score  assigned  was  the  sum  of  the  three  ratings  (3- 
12).     Inter-rater  reliability  was  .828. 

The  statistical  analysis  was  designed  to  determine  the  effect  of 
topic,  information  load  of  topic,  and  length  of  essay  on  holistic  score. 
Subject  matter  of  topics  did  not  make  a  significant  difference,  but 
information  level  effects  were  much  stronger,  and  essay  length  was  sig- 
nificantly correlated  with  score.     Level  3  essays  from  fully  specified 
topics  received  the  lowest  mean  score  and  had  the  shortest  mean  length. 
Brossell  is  careful  to  note  that  "the  differences  between  these  means 
and  those  of  the  other  two  information  levels  are  not  large  or  statis- 
tically significant.  "But  (emphasis  his)  they  are  contrary  to  what  those 
who  assume  the  superiority  of  a  full  rhetorical  specification  would 
predict"  (p.  12).     In  his  discussion  of  the  results,  Brossell  notes  that 
level  1  requires  the  writer  to  get  the  subject  in  focus  and  find  an 
organizational  scheme  while  Level  2  offers  a  definite  beginning  focus 
and  even  something  of  an  organizational  strategy.    Even  though  Level  3 
provided  a  full  rhetorical  context,  it  did  not  provide  direct  aid  for 
organizing  or  focusing  the  writing.    Brossell  reports  that  "many 
writers  failed  to  get  beyond  a  rehash  of  the  information  given,"  and 
it  "often  seemed  to  act  as  a  hindrance  rather  than  the  facilitator 
it  was  meant  to  be"  (p.  17). 

The  results  of  Brossell 's  study  have  led  Hoetker  (1981)  to  wonder 
about  the  reading  burden  of  the  longer  scenario  type  of  writing  prompt. 
He  points  out  that: 

...the  more  information  students  are  given — the  more  language 
they  have  to  process — the  greater  seem  to  be  the  opportunities 
for  creative  misreadings  and  simple  confusion.    Second,  the 
mere  information  students  are  given,  the  more  difficult  they 
seem  to  find  it  to  get  beyond  the  language  of  the  topic  to 
discover  their  own  language  (p.  10). 

Derek  Rowntree  (1977)  provides  an  insight  into  what  might  be 
happening  to  a  writer  as  the  level  of  specification  grows  in  a  given 
essay  question.    As  Rowntree  makes  the  following  set  of  questions 
progressively  more  specific,  we  can  see  that  with  each  added  constraint 
there  is  an  increasing  likelihood  of  eliciting  more  convergent  responses 
with  a  corresponding  loss  of  opportunity  for  divergent  responses. 

lo    What  aspects  of  the  political  system  of  modern  Sweden 
seem  to  you  most  worthy  of  comment? 

2.  Comment  on  the  political  stability  of  modern  Sweden* 

3.  Explain  the  political  stability  of  modern  Sweden. 

A.     Identify  and  discuss  three  factors  that  might  help 
explain  modern  Sweden's  political  stability. 
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5.     Identify  and  discuss  three  factors  that  might  help 
explain  the  emergence  of  a  stable  political  system 
in  Sweden  despite  the  massive  social  and  economic 
changes  engendered  by  processes  of  modernization. 

Rowntree  points  out  that  each  version  progresses  successively  to  restrict 
the  student's  scope  of  response.    With  this  restriction  comes  a  gain  in 
uniformity  of  response  and  comparability  of  answers,  but  there  is  also 
a  loss  of  information  about  the  possible  range  of  variability.  Since 
the  question  and  the  criteria  for  answering  are  pref ormulated ,  it  can 
no  longer  be  a  measure  of  whether  the  student  could  have  formulated  the 
question  and  decided  the  relevant  criteria.    Rowntree  concludes: 

...in  an  attempt  to  ensure  comparability  and  make  the  grading 
more  accurate,  we  may  have  ended  up  comparing  and  grading 
less  significant  abilities  in  the  student.    Macnamara's  Fallacy, 
perhaps — making  the  measurable  important  rather  than  the  impor- 
tant measurable  (p.  156). 

Flower  and  Hayes'  (1979)  process  model  of  composing  begins  with 
the  "task  environment"  ("the  world  outside  the  writer's  skin")  which 
consists  of  "the  rhetorical  situation"  ("the  specifications  of  topic 
and  audience  to  which  a  writer  must  respond")  and  "the  text  which  the 
writer  has  produced  so  far"  (pp.  90-91).    Flower  and  Hayes  (1980, 
p.  40)  discuss  "the  rhetorical  situation"  in  terms  of  "the  demands  of 
the  rhetorical  problem": 

Whatever  writers  choose  to  say  must  ultimately  conform  to 
the  structures  posed  by  their  purpose  in  writing,  their 
sense  of  audience,  and  their  projected  selves  or  imagined 
roles.     In  essence  writing  is  also  a  speech  act  and  there- 
fore subject  to  all  the  constraints  of  any  interpersonal 
performance. 

Humes  (1980,  p.  5)  says  that  "Flower  and  Hayes  consider  the  problem  to 
be  the  most  important  element  at  the  beginning  of  composing  because 
writers  solve  'only  the  problem  they  represent  to  themselves' - (p .  8)." 
Later  Humes  (p.  5)  also  mentions  that  Flower  and  Hayes  consider  dis- 
covering how  "the  rhetorical  problem  works  and  affects  the  writer  'is 
an  important  goal  for  research'   (p.  8)."    On  the  basis  of  their  own 
studies.  Flower  and  Hayes  (1980,  pp.  41-44)  point  out  the  strategies 
that  writers  employ  to  solve  the  problem  of  excessive  constraints. 
They  do  such  things  as  "throw  a  constraint  away,"  "partition  the  prob- 
lem," "draw  on  a  routine  or  well-learned  procedure,"  and  "plan."  As 
Flower  and  Hayes  put  it,  "Writers  inevitably  set  such  priorities  in  the 
way  they  define  their  Rhetorical  Problem  (e.g.,  this  is  a  letter  cO 
Aunt  Tilly,  so  you  can  safely  ignore  run-on  sentences  and  fragments; 
she  won't  mind)"  (p.  42). 

It  is  doubtless  true  thcit  Aunt  Tilly  doesn't  mind  a  few  grammati- 
cal lapses,  but  the  unforgiving  evaluator  does  mind.    Thus,  the  evalu- 
ator  needs  to  be  wary  in  specifying  audience,  for  an  injudicious  speci- 
fication may  have  syntactic  consequences  which  affect  an  assessment 
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rating.    Crowhurst  (1978,  p.  11)  found  that  a  tenth  grade  boy  writing 
in  the  mode  of  argument  produced  a  substantially  longer  mean  T-unit 
length  when  writing  for  a  teacher  audience  than  he  did  when  writing 
for  a  best  friend  audience.    The  matter  of  posing  an  audience  is  tricky, 
as  Steinberg  (1980)  notes: 

Posing  reasonable  audiences  is  not  as  simple  as  it  sounds. 
On  the  one  hand,  the  audience  should  be  specific  enough  to 
govern  what  facts  the  writer  uses  and  what  the  tone  and 
thrust  of  the  written  statements  should  be.    On  the  other, 
the  audience  should  not  be  so  exotic  as  to  invite  laughter 
or  irritation.     I  can  remember  a  period  here  at  Carnegie- 
Mellon  University  when,  in  our  zeal  for  teaching  problem 
solving  in  writing  courses,  we  posited  such  bizarre  audiences 
that  we  annoyed  our  students  instead  of  motivating  them.  In 
one  assignment,  for  example,  we  asked  students  to  write  an 
explanation  of  the  use  of  the  toothbrush  for  a  native  of 
the  Canadian  Arctic  region  who  had  never  seen  one.... In 
one  class... it  was  clear  that  the  students  were  moving 
from  amusement  to  annoyance  when  one  day  about  midsemester 
I  came  into  the  room  at  the  beginning  of  the  hour  and  saw 
on  the  board  something  like  the  following:     "Write  an 
explanation  for  a  one-armed  paper  hanger  who  is  allergic 
to  paste  about  how  he  can  paper  this  room  while  standing  on 
one  foot  without  harming  the  newly  shellacked  floor'* 
(p.  166). 

The  complexities  of  specifying  audience  in  writing  assignments  are 
analyzed  in  detail  by  Long  (1980)  and  by  Park  (1982)  .    Long  argues  that 
the  traditional  ways  of  specifying  audience  in  writing  exercises  by 
listing  observable  physical  or  occupational  characteristics  (white 
male  university  administrator)  rests  on  the  false  assumption  that 
"people  sharing  certain  superficial  qualities  are  alike  in  all  other 
respects"  (p.  223).     It  is  a  practice  that  he  thinks  would  be  called 
"stereotyping"  in  any  other  context.    Long,  therefore,  calls  for  a 
redefinition  of  audience  and  the  application  of  literary  theory  to  a 
refinement  of  the  concept.    He  turns  to  Walter  Ong's  concept  of  "the 
audience  as  a  fiction"  for  his  new  model.    Long  quotes  Ong  who  says 
"the  writer  must  construct  in  his  imagination,  clearly  or  vaguely,  an 
audience  cast  in  some  sort  of  role — entertainment  seekers,  reflective 
sharers  of  experience, .. .and  so  on.... A  reader  has  to  play  the  role  in 
which  the  author  has  cast  him,  which  seldom  coincides  with  his  role  in 
the  rest  of  life."    This  view  of  the  audience  as  a  created  fiction 
requires  the  writer  to  posit  a  new  set  of  questions.    Long  says. 

Rather  than  beginning  with  the  traditional  question,  "who 
is  my  audience?"  we  now  begin  with,  "who  do  I  want  my 
audience  to  be?"    Rather  than  encouraging  a  superficial, 
stereotyped  view  of  the  reader,  we  are  asking  the  student 
to  begin  with  a  statement  about  the  audience  she  wants  to 
create. .. .This  leads  directly  to  question  of  method:  what 
distance  between  reader  and  subject  should  be  established? 
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What  of  diction  and  the  creation  of  tone?    What  pieces  of 
information  do  I  want  the  reader  to  take  for  granted?, 
etc.  (p.  225) . 

Douglas  Park  (1982)  offers  a  lengthy,  astute  analysis  of  the 
several  meanings  of  "audience/'    He  finds  a  number  of  conceptual  traps 
in  the  way  audience  is  typically  used.    When  writers  are  told  "Consider 
your  audience,"  it  is  too  simple  to  take  this  to  mean  concentrate  upon 
some  particular  person  or  persons.    Writers  attend  to  a  number  of  dif- 
ferent kinds  of  issues  when  they  think  about  audience.     According  to 
Park  (p.  249)  the  meanings  of  audience  tend  to  move  in  two  directions: 
one  movement  is  toward  actual  people  external  to  the  text;  the  other 
movement  is  toward  the  text  itself  and  the  audience  implied  there.  But, 
however  real  the  readers  are  outside  theitext,  "the  writer  must  create  a 
context  into  which  readers  may  enter  and  to  varying  degrees  become  the 
audience  that  is  implied  there"  (p.  249). 

Park  finds  the  two  general  meanings  of  audience  dividing  into  four 
more  specific  meanings  (p.  250): 

1.  Anyone  who  happens  to  listen  to  or  read  a  given  discourse: 
"The  audience  applauded." 

2.  External  readers  or  listeners  as  they  are  involved  in  the 
rhetorical  situation:     "The  writer  misjudged  his  audience." 

3.  The  set  of  conceptions  or  awareness  in  the  writer's 
consciousness  that  shape  the  discourse  as  something  to 
be  read  or  heard:     "WTiat  audience  do  you  have  in  mind?'* 
represents  the  shorthand  version  of  this  set  of  aware- 
nesses. 

4.  An  ideal  conception  shadowed  forth  in  the  way  the 
discourse  itself  defines  and  creates  contexts  for  readers: 
"What  does  this  paragraph  suggest  about  audience?" 

The  latter  two  conceptions  are  most  important  for  teachers  or  for  per- 
sons interested  in  discourse. 

After  considerable  elaboration  upon  these  several  meanings  of 
audience.  Park  eventually  considers  the  implications  of  his  analysis 
for  the  teaching  of  audience.    He  believes  teachers  depend  too  heavily 
on  the  concrete  image  of  audience  as  readers  external  to  the  text.  He 
then  comments  on  the  strategy  of  using  members  of  the  class  or  the 
teacher  as  the  audience  for  student  papers: 

The  former  strategy,  of  course,  means  that  students  write 
knowing  that  their  papers  will  be  "published"  in  the 
classroom,    '^nis  practice  has  obvious  powerful  effects  on 
how  students  see  the  act  of  writing,  but  it  can  be  said 
to  provide  an  audience  only  in  the  commonly  used  sense 
of  external  to  listener    or  readers.     Students*  reading  of 
one  another's  writing  does  not  provide  that  crucial 


ERJC 


ingredient,  people  rhetorically  involved.    The  student 
writing  for  members  of  the  class  still  has  the  problem 
of  finding  or  inventing  appropriate  rhetorical  contexts. 
In  fact,  useful  as  this  strategy  is,  it  may  also  create 
problems. 

The  awareness  of  specific  critical  readers,  whether  they  be  students 
or  the  teacher,  may  inhibit  and  complicate  rather  than  simplify  the 
problem  of  dealing  with  audience. 

Park  ultimately  says,  "the  fact  is  that  most  of  the  time  we  want 
students  to  learn  to  write  for  a  'general'  audience."    That  is  to  say  we 
want  them  to  write  in  relatively  unstructured  situations  where  little  is 
given  in  the  way  of  context  and  much  remains  to  be  invented  by  the  writer." 
Park  believes  teachers  need  to  be  able  to  break  audience  problems  down 
into  specific  issues  and  strategies.    The  strategies  must  "take  shape 
against  the  background  of  the  conventions  appropriate  for  given  kinds  of 
writing.     Much  of  the  time  it  is  not  possible  to  separate  a  sense  of 
audience  from  a  sense  of  genre  and  convention. .. .Probably  writers  [who] 
come  to  have  an  intimate  sense  of  audience  as  convention  [do  so]  by  being 
readers  of  that  kind  of  prose.     I  doubt  if  it  can  be  taught  very  directly 
or  very  quickly"  (p.  256).    Ultimately,  a  clear  understanding  of  audience 
must  grow  from  a  clear  understanding  of  the  kinds  of  discourse  and  their 
purpose  in  society. 

It  can  be  seen  from  these  two  analyses  of  the  '"meanings"  of  the 
term  "audience"  that  it  is  a  conceptually  complex  rhetorical  construct. 
Thus,  when  researchers  such  as  Scardemaliaj  Bracewell,  and  Bereiter 
(1978)  suggest  "that  assignments  that  specify  a  particular  audience  in 
some  detail  act  as  important  cues  to  writers  because  they  help  them 
'decenter'  from  their  content  or  information  and  adapt  their  knowledge 
to  a  reader"  (quoted  in  Flower  and  Hayes,  1980,  p.  A5),  one  cannot  be 
sure  exactly  what  this  finding  means  in  terms  of  what  was  actually 
specified.    There  needs  to  be  a  good  deal  more  critical  analyses  of 
different  kinds  of  prose  and  more  research  into  the  composing  process 
before  we  can  be  sure  how  best  to  guide  test-makers  in  designing  writing 
prompts. 

Perhaps  the  writing  task  can  be  simplified  conceptually,  as  Linda 
Flower  proposes,  into  understanding  the  distinction  between  what  she 
calls  "writer-based"  prose  and  "reader-based"  prose.    As  she  puts  it, 
"Good  writers  know  how  to  transform  writer-based  prose  (which  works 
well  for  them)  into  reader-based  prose  (which  works  for  their  readers 
as  well)"  (Fl  ower ,  1981,  p*  1A4) .    Flower  deals  in  the  development  of 
process  strategies  "to  create  a  momentary  common  ground  between  the 
reader  and  the  writer"  (p.  122).     In  her  view,  then,  the  accomplishment 
of  reader-based  prose  requires  developing  a  critical  awareness  of 
features  of  the  intended  reader  such  as  his  knowledge,  attitudes,  and 
needs.     It  is  necessary  to  go  even  beyond  this  base,  however,  for  con- 
temporary reading  theory  posits  a  "creative"  reader  activit  in  the  making 
of  meaning,  but  a  reader  with  certain  limitations  of  short-term  memory 
and  certain  expectations  for  the  structural  pattern  of  the  text  being 
read.     Thus,  the  writer  needs  not  only  to  know  his  reader,  but  he  needs 
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to  aid  him  by  providing  a  framework  or  a  context  for  new  ideas  (Flower, 
1981,  pp.  121-142,  passim). 

The  evidence  just  recounted  here  from  rhetorical  theory  (Long, 
1980;  Park,  1982)  and  the  research  into  the  cognitive  processes  of 
writing  (Flower  and  Hayes,  1979;  Humes,  1981;  Flower,  1981)  make  moot 
the  wisdom  of  declaring  a  particular  role  and  audience  for  writers.  To 
require'  a  writer  to  assume  the  role  of  an  advertising  copy  writer  and 
prepare  an  advertising  campaign  for  the  executives  of  Acme  Beer  (topic 
idea  from  Sarah  Freedman)  presupposes  prior  knowledge  on  the  part  of 
the  writer  of  the  nature  of  the  copywriting  role,  advertisement  and 
report  genre,  and  of  Acme  Beer  executives  as  prospective  readers  of 
the  advertising  campaign  material.    An  exceedingly  complex  task  becomes 
even  more  convoluted  as  the  writer  realizes  that  in  the  real-world 
context  of  the  assessment  situation,  his  only  real  reader(s)  will  be 
the  evaluator ( s) . 

5.    Contextual  Influences  on  Assessment,    We  are  just  beginning 
to  understand  the  nature  of  context  as  an  encompassing  framework  around 
the  process  of  writing  and  assessment.    Staton  (1981,  p.  vil) ,  reminds 
U8  that  "there  Is  research  to  suggest  that  the  greatest  failure  of 
writing  instruction  and  assessment  comes  in  not  creating  conditions 
under  which  students  would  have  reason  to  compose  a  text  (Graves,  1978, 
1979)."    Roger  Shuy  (1981)  raises  a  number  of  questions  about  assessment 
and  the  constraining  conditions  imposed  upon  the  writer.    He  concludes 
that 

What  we  are  assessing  is  not  writers'  ability  to  think, 
organize,  sequence,  explain,  persuade,  narrate,  or  describe, 
but  rather  their  ability  to  do  these  things  under  the  special 
set  of  circumstances  created  for  the  writing  assessment  (p.  170). 

Shuy  chooses  an  absurd  metaphor  to  convey  the  Idea  that  assessment  measures 
only  the  ability  to  do  a  specific  task  under  constraining  conditions.  Our 
approaches  to  writing  assessment  can  be  likened  to  an  approach  to  the 
assessment  of  walking  which  assesses  "the  ability  to  walk  on  a  slippery 
pavement  with  a  broken  toe  and  high-heeled  shoes"  (p.  171).    We  would  not 
necessarily  learn  anything  about  ability  to  write  or  to  walk  in  this 
approach  tossessment,  according  to  Shuy. 

Michael  Clark  (1980)  makes  an  astute  analysis  of  the  power  relation 
that  holds  between  participants  in  an  assessment  situation.    To  portray 
this  condition  he  draws  on  the  famous  work  of  William  Labov  who  examined 
the  interviewing  techniques  of  Interviewers  whose  results  led  to  classi- 
fication of  their  subject  as  "linguistically  deprived."    He  quotes  two 
key  passages  from  Labov  which  he  feels  speak  equally  well  to  the 
situation  in  writing  assessment,  although  Labov's  observations  apply 
directly  to  an  interview.     First,  Labov  points  out  the  origins  of  a 
subjert's  defensive  maneuvers  In  an  interview  situation: 

The  child  is  in  an  asymmetrical  situation  where  anything  he 
says  can  literally  be  held  against  him.    He  has  learned  a 
number  of  devices  to  avoid  saying  anything  in  the  situation. 
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and  he  works  very  hard  to  achieve  this  end..  ..If  one 
takes  this  interview  as  a  measure  of  the  verbal  capacity 
of  the  child,  it  must  be  at  his  capacity  to  defend  him- 
self in  a  hostile  and  threatening  situation  (1969,  p.  8) 
(quoted  in  Clark,  1980,  p.  130). 

In  a  second  passage  quoted  by  Clark,  Labov  provides  the  underlying 
interpretation  of  an  interviewer's  question  such  as  "What  do  you  Think 
of  capital  punishment?" 

the  speaker's  interpretation  of  these  requests,  and  the 
action  he  believes  appropriate  in  response  is  completely 
uncontrolled.    One  can  view  these  test  stimuli  as  requests 
for  information,  commands  for  action,  as  threats  of  punish- 
ment, or  as  meaningless  sequences  of  words.    They  are 
probably  intended  as  something  altogether  different:  as 
requests  for  display  (1969,  p.  23)   (quoted  in  Clark,  1980, 
p.  130). 

Although  the  actual  stimulus  question  may  be  the  same  for  each  respon- 
dent, virtually  all  of  the  other  variables  of  the  connnunication  con- 
text may  differ  from  respondent  to  respondent.     Consequently,  says 
Clark,  "To  judge  all  the  samples  elicited  from  the  interviews  by  the 
same  criteria  is  therefore  to  render  the  evaluations  meaningless" 
(p.  131). 

Clark  declares  that  "any  evaluation  of  speech  or  writing  that 
ignores  the  extraverbal  ' sociolinguistic'  feature  of  the  context  in 
which  the  language  was  elicited  is  irrelevant  as  a  measure  of  communi- 
cative competence.     Furthermore,  any  assessment  of  language  that  pur- 
ports to  measure  verbal  capacity  outside  of  a  specific  context— 'Good 
Writing' — is  therefore  either  deliberately  misleading  its  users  or  is 
looking  for  something  that  is  precluded  by  the  test  itself"  (p.  131). 
After  advancing  this  line  of  argument.  Clark  makes  the  claim  that  the 
situation  in  Freshman  English  and  in  any  of  the  evaluative  contexts  is 
like  that  described  above,  and  "shows  our  assignments  and  evaluations 
often  proceed  from  the  same  false  assumptions  about  the  context-free 
nature  of  language  acts"  (p.  131).    Clark  then  proceeds  to  describe  an 
approach  to  assessment  examinations  of  incoming  students  at  the^  Univer- 
sity of  Michigan  which  he  says  is  designed  and  read  taking  into  account 
some  of  the  insight  summarized  above.    No  examination  question  Is  pro- 
vided as  a  sample,  but  the  pattern  is  described.     It  would  be  mis- 
leading to  summarize  the  examination  procedures  without  providing 
Clark's  interpretation  of  them,  and  so  that  will  not  be  attempted  here. 
But  one  does  have  to  wonder  to  what  extent  the  University  of  Michigan 
assessment  procedure,  though  much  refined,  escapes  the  dec ontextualizing 
processes  of  other  examinations.    One  also  has  to  wonder  how  the 
students  in  the  Michigan  test  escape  "the  role  established  for  them 
by  the  context  of  the  test"  any  more  readily  than  the  participants  In 
any  other  test  situation. 
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There  is  a  confirmation  of  aspects  of  "sociolinguistic"  variables 
of  the  assessment  situation  in  Pianko's  (1979)  study  of  the  processes 
of  college  freshman  writers.    Pianko  finds  a  number  of  "interesting 
paradoxes"  in  the  methods  currently  used  for  teaching  writing. 

In  the  first  instance,  students  do  not  view  writing  which 
has  the  context  specifically  set  by  the  teacher  and  which 
must  be  completed  within  the  constraints  of  a  class  meeting 
as  an  activity  that  is  worth  committing  themselves  to.  The 
limitations  placed  by  the  typical  school  writing  activity 
negate  the  possibility  for  greater  elaboration,  commitment, 
and  concern.    Yet  many  instructors  insist  that  the  most 
effective  way  to  evaluate  students*  writing  abilities  is 
to  have  the  writing  controlled  for  topic,  place,  and  time. 
According  to  the  students  in  this  study,  such  a  writing 
activity  does  not  permit  sufficient  time  for  them  to  re- 
group their  energies  and  thoughts;  therefore,  they  merely 
attempt  to  complete  the  assignment  in  some  expedient 
fashion  and  "give  the  teachers*  what  they  want."    So,  in 
fact,  what  writing  teachers  are  actually  evaluating  is  how 
well  students  follow  instructions,  not  how  well  they 
write  (p.  18). 

Pianko's  work  appears  to  offer  empirical  validation  of  the  socio- 
linguistic  insights  of  Labov,  Clark,  and  others  mentioned  above. 

In  1979-80  Clark  and  Florio  (1982)  undertoo,   a  naturalistic 
study  of  schooling  and  the  acquisition  of  literary.    They  sought 
to  document  how  two  teachers  and  their  respective  students  produce 
occasions  for  writing  by  means  of  interaction,  to  describe  the 
process  of  teacher  planning  for  writing  instruction,  to  develop  a 
typology  of  the  diverse  occasions  for  writing  in  each  classroom, 
and  to  examine  these  occasions  as  contexts  for  writing  that  may 
make  differing  cognitive  and  social  demands  on  the  students  as 
writers  (p.  27).     It  is  the  latter  purpose  of  this  study  which  is 
of  interest  here.     Clark  and  Florio  identify  four  broad  functions 
of  writing  in  a  second/third  grade  classroom  and  in  a  sixth  grade 
classroom  (p.  159): 

1.  writing  to  participate  In  community, 

2.  writing  to  know  oneself  and  others, 

3.  writing  to  occupy  free  time,  and 

A.    writing  to  demonstrate  academic  competence. 

We  will  consider  here  only  "writing  to  demonstrate  academic 
competence,"  for  we  seem  to  have  here  a  clear  case  of  what  Labov 
called  the  "display  function"  of  an  assessment.    According  to  Clark 
and  Florio,  in  classrooms,  academic  writing  "is  a  teacher-initiated 
activity  and  it  is  the  only  type  of  writing  that  received  formal 
evaluation  from  the  teacher..." 

It  is  the  only  type  of  writing  that  was  typically  both 
composed  and  formatted  by  an  outside  third  party — the 
publisher.     This  fact^  so  commonplace  in  classrooms  in 
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Clark  and  Florio  conclude  "that  ambiguity  about  the  audience  of  a 
student  writing  activity  can  affect  both  the  interpretation  of  the 
task  and  the  students'  performance  of  the  writing  in  diverse  ways" 
(p..  67). 

Further  insight  into  the  school  as  a  context  for  writing  is 
reflected  in  the  Applebee  (1980)  report  on  writing  ir.  the  secondary 
school.    This  study  reports  results  from  classroom  observations 
of  a  year's  writing  assignments  in  a  comprehensive  city  high  school 
and  in  a  university-associated  laboratory  school  and  from  a  question- 
naire submitted  to  a  national  sample  of  teachers  in  six  subject 
areas. 

The  observational  study  revealed  that  an  average  of  44  percent 
of  observed  lesson  time  was  involved  with  writing  activities.  . 
Mechanical  uses  of  writing  (short  answer,  f ill-in-the-blank  tests) 
occurred  24  percent  ol  the  Lime»  uoce  taking  17  percent,  and  writiag 
of  paragraph  length  or  longer  occurred  3  percent  of  the  observed 
time.     Homework  assignments  of  at  least  a  paragraph  length  occured 
3  percent  of  the  time.    The  writing  of  paragraph  length  occurred 
most  frequently  in  English  classes,  averaging  10  percent  of  lesson 
time.    The  most  prevalent  writing  task  was  classified  as  requiring 
informational  writing;  imaginative  uses  were  limited  for  the  most 
part  to  English  classes,  but  reported  by  less  than  half  of  the  student 

The  national  survey  of  teachers  in  all  subject  areas  indicated 
that  they  made  frequent  use  of  some  writing-related  activities  but 
these  were  dominated  by  note  taking  and  short-answer  responses. 
Paragraph  length  writing  was  reported  in  27  percent  of  classes  at 
grade  nine  and  in  36  percent  at  grade  eleven.    The  writing  samples 
supplied  reflected  informational  uses  of  writing  in  85  percent  of 
the  cases.      The  teacher  in  the  role  of  examiner  was  the  prime 
audience  for  student  writing  in  all  subject  areas.    Only  10  percent 
of  the  teachers  reported  that  the  writing  was  read  by  other  students. 
Of  the  writing  samples  submitted  by  surveyed  teachers,  88  percent 
were  addressed  to  the  teacher  as  the  primary  audience.  Generally, 
48  percent  of  the  sample  was  informational  writing  addressed  to  the 
teacher  as  examiner. 

The  consequences  of  the  emphasis  on  the  more  mechanical  forms 
of  writing  required  across  the  secondary  curriculum  are  described 
by  Applebee  (p.  141): 

Analyzed  as  writing  activities,  such  tasks  are  characterized 
by  a  separation  of  the  problem  of  constructing  coherent 
text  in  a  language  appropriate  to  the  subject  areas  from 
the  problem  of  remembering  subject  area  information  and 
concepts.     Essentially  the  teacher  takes  over  all  the 
difficulties  inherent  in  using  language  appropriate  to  a 
subject  area — including  much  of  the  specialized  vocabulary 
and  rules  of  procedure  which  are  embedded  in  the  text — and 
leaves  the  student  only  the  task  of  mechanically  "slotting- 
in"  the  missing  information  (p.  141). 
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our  culture,  was  at  first  taken  for  granted  by  the 
researchers.    However,  when  students  were  asked  to 
sort  and  talk  about  their  written  work,  they  consis- 
tently grouped  worksheets  and  workbook  pages  together 
because  they  were  produced  "by  machine"  (p.  165). 

These  academic  performance  writing  activities  engaged  students  in  a 
variety  of  discourse  functions  including  explanation  and  description, 
but  these  activities  were  not  generated  or  ultimately  controlled  by 
the  students: 

the  topic  was  constrained,  the  format  of  the  writing 
was  limited,  and  the  function  of  the  writing  was  pre- 
determined by  others  (p.  166). 

Clark  and  Florio  provide  an  interesting  example  of  how  a  teacher- 
imposed  task  that  is  intended  to  stimulate  private,  free  writing 
(fulfilling  the  writing-to-know-oneself  function)  is  actually,  in 
effect    interpreted  by  the  students  in  a  way  unintended  by  the 
teacher  and  becomes,  in  effect,  writing  to  display  competence.  One 
teacher  set  up  an  activity  called  "Diary  Time"  which  was  Intended 
to  be  mainly  private.     Students  were  to  be  free  of  concerns  about 
audience  and  form:     they  were  to  write  their  own  thoughts.    But  as 
it  worked  out.  Diary  Time  had  some  features  that  made  it  similar  to 
more  formal  types  of  school  writing.     Clark  and  Florio  describe  the 
Diary  Time  writing  situation  (p.  65)  u. 

The  facts  that  Diary  Time  was  a  teacher-imposed  task,  that 
the  teacher  knew  and  taught  about  "the  rules"  of  diary 
keeping,  that  the  teacher  read  aloud  «  part  of  her  own 
first  journal  entry,  and  that  the  teacher  collected  and 
stored  the  diaries  in  her  filing  cabinet  all  supported  the 
idea  of  the  "teacher  as  audience."    Student  concerns  about 
accurate  spelling,  questions  about  h»  the  teacher  will 
correct  them,"  and  teacher  assistance  for  those  who  could 
not  think  of  what  to  write  constitute  additional  evidence- 
of  teacher  as  audience. 

Clark  and  Florio  found  this  to  be  an  instance  of  an  assignment  that 
sent  a  "mixed  message"  to  the  students  causing  confusion  In  their 
efforts  to  accomplish  it.    They  observed  three  types  of  student 
response  (p.  66-67): 

1      Some  treated  the  Diary  Time  as  "just  another  school 

writing  task"  and  wrote  what  they  thought  would  please 
the  teacher. 

2.  Some  treated  the  Diary  Time  as  a  reflective  writing  task 
where  the  usual  "rules"  of  school  writing  were  suspended. 

3.  Some  wrote  nothing  and  waited  for  the  teacher  to  help 
them  think  of  what  to  write  and  thereby  remove  the 
ambiguity  from  this  new  type  of  writing  task. 
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One  of  the  major  problems  with  an  overemphasis  on 
mechanical  writing  tasks  is  that  the  students  may  never 
learn  to  use  such  resources  on  their  own,  relying  instead 
upon  the  structure  or  scaffold  that  the  teacher  has 
provided  (p.  143). 

•  •  • 

Because  the  emphasis  is  on  specific  items  of  information, 
rather  than  on  the  way  those  items  are  integrated  and 
presented  in  coherent  prose,  such  writing  situations 
provide  little  opportunity  for  instruction  that  might  help 
students  develop  specific  writing  skills.    For  learning 
to  write  well,  the  most  effective  vnriting  situation  will 
be  one  in  which  the  effectiveness  of  the  writing  matters— 
where  the  student  can  savor  the  success  of  having  presented 
a  convincing  argument,  or  struggle  with  the  problems  of 
having  failed  to  do  so  (pp.  143-144). 

What  emerges  from  the  Applebee  study  that  has  relevance  to  the  assess- 
ment condition  is  that  students  live  in  a  classroom  setting  that 
offers  little  opportunity  to  practice  the  writing  skills  upon  which 
they  may  be  formally  assessed.    Much  of  the  classroom  writing  that 
the  iudents  do  is  assigned  in  a  test  situation  rather  than  in  an 
instructional  one.    The  test  situation  itself  tends  to  be  undemanding 
of  integrative  compositional  processes  because  the  dominant  test  forms 
require  only  the  "slotting- In"  of  information  or  short  answers  which 
do  not  test  one's  mettle  at  structuring  an  idea.    The  informational 
writing  that  constitutes  the  principal  type  of  assignment  tends  to 
require  only  relatively  isolated  bits  of  information,  often  only  in 
the  context  of  highly  prestructured  tasks  (p.  142). 

These  teacher~as-examiner  prestructured  tasks  would  seem  to 
constitute  scant  preparation  for  the  writing  assessment  tasks  fre- 
quently demanded.    These  teacher-prepared  texts  with  gaps  to  be  filled 
in  seem  to  call  for  a  different  sort  of  cognitive  ability  than'  is 
required  for  creating  and  monitoring  the  development  of  one's  own 
text  in  an  impromptu  text  produced  for  a  writing  test  (see  Bartlett, 
1981,  p.  23).     The  teacher's  own  writing  assignments  and  assessment 
tasks  form  an  unobtrusive  context  for  what  later  happens  in  the 
public  examination,  whether  it  occurs  at  the  school  level  or  in  a 
national  sample.     Logic  suggests  that  there  should  be  preparatory 
work  for  what  is  to  be  assessed  and  that  the  examination  itself 
should  not  constitute  the  only  practice  of  particular  task  demands 
and  discourse  forms. 
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WHAT   COUNTS   AS  RESEARCH? 
PROCEDURES   OF  THE  STUDY 


Knovledge  Without  Numbers 

It  has  long  been  a  requirement   in  educational  research 
that   one   design  studies  which  would  generate  statistics  in 
order   to   formulate  conclusions   that   the  general   community  of 
educational  researchers  would  consider  binding.  Grounded 
speculation  which  seemed  to   follow  logically   from  evidence 
did  not  count  as  research.      Common  sense  was  suspect.  But 
eventually   some   sociologists,   cognitive  scientists,  and 
social  theorists  began  to  question  our  narrowly-conceived 
definitions  of  research.     One  of  these   sociologists  stated 
the   issue   succinctly  in  relation  to  the  study  of  human 
affairs:      "...not   everything   that   cBn  be   counted   counts,  and 
t   everything   that   counts   can  be   counted"    (Cameron,  1963, 


p.  13).  More  recently,  at  the  annual  cognitive  sciences 
conference  held  at  the  University  of  California,  Berkele 
the   fall  of  1981,  Robert  Abelson  of  Yale's  Artificial 


con 
the 

Intelligence  Project,    observed  that 
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It   is   difficult   to   formalize   aspects  of  human  thought 
which  are  variable,   disorderly,   and  seemingly 
irrational,   or  to  build  tightly  principled  models 
of   realistic  language  processing  in  messy  natural 
domains,    (p.  1) 

Abelson  went  on  to  characterize   the  clash  of  research  tradi- 
tions as   being  between   the   "neats"   and   the  "scruffies  -in 
other  words,  between  those   "whose   canons  of  hard  science 
dictate  a  strategy  of   the  isolation  of   idealized  subsystems 
which  can  be  model'ed  with  elegant  productive  formalisms  and 
those     other   scientists  who   seek  to   deal  with  'the  rough  and 
tumble  of   life   as  it   comes"    (p.  D- 

Something  of   this   same   type   of   ferment  must  have  been 
evident   in  the  September  1980  NIE-FIPSE  Grantee  Workshop^^ 
that   convened   in  California,    for   in  his  "Closing  Remarks^^ 
Roger  Shuy  listed  several  of   the   currents  of  uneasiness 
that  he  had  observed  at   the  meeting.     Among  them  was  the 
uneasiness   "about   the  way  we  have  used  data   in  the  past. 
I    think  maybe   the  big  battles  in  many   disciplines   are  the 
battles   over  what   constitutes  good   data. ...we  still  have  a 
kind  of   uneasiness  about   data,   how  to  use   it,   where   it  comes 
fro,.,  what   it  amounts   to"      (p.   166).      Shuy  went  on  to  remark: 

The   old  model  of   theory   going  to   research  going  to 
practice   is  no   longer   viable.      Rather,    the  view 
presented  here  was  more  like   the  view  presented 
of  writing— a  recursive,     iterative  model,  with 
theory,   research  and  practice   interacting   (p.    IbS) . 
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Meanwhile,    in  her  "Welcome"  at   the  same  conference, 
Marcia  Farr   (Whiteoan)   addressed  the  question  of  methodology 
in  writing  research: 

We   are  devising  our   own  tools,    choosing  from  among 

a  variety  of   social-science  disciplines  and  approaches 

as  we   go   along.     We  must   create   structure   out   of  what 

we  find--out   of  data  Writing  is  a  human  activity; 

as  such,   what  we  really  need  to  do   in  order   to  get 
valid  information  is  pick  our  way  along  and  eclectically 
use  methodology  from  here  and   there,   selecting  tech- 
niques  to  help  us   learn  what  we  want  to  know.  Being 
eclectic,   of   course,   does  not  preclude  being  rigorous 
in  our   thinking   (p.  13). 

And   later   in   this   same   "Welcome"  Marcia   Farr  added: 

Often  researchers   get   too   far   from   the  social 
contexts   in  which  their  work  could  be  useful,  and 
their  work  becomes   so  esoteric   that   it  requires 
much   further  work  to  "translate"   the  findings 
into  practice.      It   is   also   important   for   those  who 
teach  writing  every   day   to  participate   in  formulating 
the  research   quest  ions  ...  (p .  13). 

The   eclectic     research  strategies   employed   in  the  conduct 
of   the  various   phases  of   this  Writing  Assessment   Project  find 
their   authority   in   spokespersons   from  the  research  community 
such   as   those   identified   above   as  well   as   in   those   from  the 
strongly   articulated  views   of   a   group   of   British  educationists 
(Stenhouse.   1981;   Bassey,   1981;   Guy,   1975;  Volpe,  1981; 
Hogan,    1980;    Elvin,    1975).      It   is  not   appropriate  here  to 
review  the  methodological  issues  addressed  by  all  of  these 
writers,   but   it  does  help  the  case  being  developed  here  to 
consider   the   position   taken  by  Lawrence   Stenhouse.  ?irst, 
Stenhouse   (as  quoted  by  Bassey,   1981,   p.   74)   suggests  that 

Our   problem  is   to   find  approaches   to  research  which 
produce   theory  which   is  of  use  both  to  practitioners 
of   education   and   to  practitioners   of  educational 
research  and  which  enables  both   to  act   in  the  light 
of   systematic  Intelligence. 

Bassey  continues   to  quote  Stenhouse: 

He   goes   on  to  assert   that  "the  most  important 
distinction   in  educational  research  at  this  moment 
is   that  between  the  study  of   samples  and   the  study 
of   cases"... it   is  clear  that  he  expects  both  the 
study  of   samples  and  the  study  of   cases  to   lead  to 
generalizations. 

Stenhouse  aistinguishes  between  predictive  generali- 
zations,  which  arise   from  the   study  of   samples  and 
are  the   form  in  which  data  are  accumulated  in 
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science,   and  retrospective  generalizations, 
which   can  eventually  arise   from  the  analysis 
of   case  studies   and  are   the   form  in  which  data 
are   accumulated   in  history.      He  writes  "while 
the  predictive   generalizations   claim  to  super- 
sede  the  need   for  ii:dividual  judgment,  retro- 
spective  generalizations   seek  to  strengthen 
individual  judgment  where   it   cannot  be  super- 
seded   (p .    75)  . 

It   is   this   power  of   the  "retrospective  generalization" 
that   is   of   particular   interest   in  this   study,   for  Stenhouse 
offers  a  way  of  making  respectable   the  use  of   common  sense 
and  practical  knowledge  in  educational   communities.     In  a 
significant   article,   "What   Counts  as  Research?"  (1981), 
Stenhouse   discusses   the  relevance  of  both  science  and  history 
to   practice,   but   it   is  his   account  of   the  significance  of 
history-as-resear ch  method   that   is  of  prime   importance  in 
describing   the  rationale  behind  our  procedures  for  consensually 
validating   the  report   that   constitutes  Part   II  of   this  chapter. 

Stenhouse  discusses  how  his tory-as-r esear ch-me thod  helps 
define   the   conditions  for   future   action  by   summarizing  exper- 
ience  and   suggesting  considerations   to   take   into  account  as 
we  make  judgments  about  how  to   act.     Thus,   judgments  of, 
relevance   about   acting  in  any   given  case  can  be  founded  on 
"stock-taking,"     state-of-the-art  reports,   which  Stenhouse 
conceives  of   as   "contemporary  histories"    (p.    105).  History 
summarizes   the   experience  of  action  in  such  a  way   as  "to 
strengthen  judgment   and  revision   of   judgment   in  the  planning 
of   acts."     Stenhouse   further  points  out   that   the  historian 
identifies   issues  Just  as   often  by  ambiguity  as  by  stating 
hypotheses.     Further,   the  historian  takes  most  of  his  terms 
from  his   subjects,    thus  "One  great  strength  of  history  is 
that   its  vocabulary   is  accessible  to   those  who  are  iTiterested 
in   the   toipc  under   discussion"    (p.  108). 

Stenhouse   respects  the  social  science   experimental  and 

analytic  models,   but  he  finds   little   in  the  reports  of  "mean 

results"   that   is   of   use  in  the   fields  of   action  in  classrooms 

wher ejteacher s  must  make  individual  decisions.     Two  more 

quotations  will   suffice  to   define  Stenhouse's  position  and 

advance  the   theoretical  justification  for   several  of   the  types 

of   studies   conducted   in  this  project.     Stenhouse  summarizes: 

In  short,    it   seems   that,   while  social   science  applied 
to  education  can  produce   results  which  help  us  to 
understand   the   ground  rules  of  action,    it  cannot 
provide   the  basis   for   a   technology  of   teaching  which 
offers  reliable   guidance   to   the   teacher.  Predictions 
based  upon   statistical   levels  of   confidence  are 
applicable   to  action  only  when  the   same  treatment 
must   be   given   throughout   the  entire  population.  This 
condition  does  not   apply   in  education.      It   is  the 
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teacher's   task  to  differentiate  treatments. 


The  portrayal  of   cases  offers   to  inform  the 
judgment  of   actors--the  administrators,  teachers, 
pupils   or  par ents~-r ather   after   the  manner  of 
history,   by  opening  the  research  accounts  to 
recognition   and  comparison  and  hence   to  criticism 
in   the  light   of  experience.     Such  a  refinement 
of   experienced  practical  Judgment   eludes  the 
psycho-statistical  model  which  strips   the  data 
of   recognizable  characteristics  and   content,  and 
presents   "findings"  or  "results,"  which  are 
accessible   to  criticism  only  by  replication  or 
by   technical   attack  on   the  design  or   conduct  of 
the   research   (p.  107). 

A  Collaborative  Model   for  Validating   the   "Theory  of  Idealized 
Practice" 

So  much  of   educational  research  deals  with  data   that  is 
so  highly  abstracted,    so  remote   in   the   language  and   form  of 
its   expression,    that   the   teachers  and  students  who  were  the 
original   creators   of   the  data  cannot  recognize  any  human 
connection   in  the  reported  results.     The  results  do  not 
seem  to  relate   to  particular  human  beings   interacting  in 
particular   contexts.     Thus,   we   sought  a  method  of  investi- 
gation that  might   tap   the  store  of  wisdom  possessed  by 
teachers   and   test-makers  practiced   in  the   creation  of  writing 
prompts  and   the   conduct   of  holistic  writing  assessment.  Since 
we  wanted   a  method   that   preserved   the  identity  of  experienced 
practical   judgments  about   the   design  of  writing  prompts,  we 
could  not   turn   to  any  of   the   conventional   statistical  survey 
research  models.     We  found  no  guidance   in  any  extant  work  on 
educational  research  methodology,   so,   as  Marcia  Farr  predicted 
might  be  necessary,  we   ended  up   inventing  our   own  procedure 
for   gathering  our  personal  and   professional  knowledge  and 
exposing   it   to   critical  scrutiny. 

i« 

Catharine  Keech  as   Informant.      In  effect,   we  used  a 
teacher-resear cher^      Catharine  Keech,   as  an  informant  to 
compose  a  document  reflecting  her  perception  of   the  essential 
procedures   to  follow  in  developing  writing  prompts   for  holis- 
tic assessment.     Ms.   Keech  brought   to  her   task  the  dual  per- 
spective pf  a  classroom  teacher  and  an  experienced  researcher 
in  writing  evaluation.      Prior   to  entering  graduate  study  at 
Berkeley,   Ms.   Keech  had   taught   for   twelve  years,   ranging  from 
elementary-  grades   through  secondary  school  and  college.  In 
1977,   Ms.   Keech  went   to  work  for   the  Evaluation  Unit  funded 
by   the   Carnegie   Foundation  to   study   the  effectiveness   of  the 
Bay  Area  Writing  Project.     This  Evaluation  Unit,   under  the 
direction  of  Michael  Scriven,   assigned  Ms.   Keech  as   a  research 
assistant   to  collect  background   information  on  how  writing  is 
evaluated   in  this   country  in  order   to  approach  the  evaluation 

9«  ,  ■ 


of   the  Bay  Area  Writing  Project.     During  the  course  of  her 
investigation,   Ms.   Keech  consulted  Evans  Alloway  of  Educa- 
tional Testing  Service*   Jce  Steele  of   the  American  College 
Testing  Service,   Rex  Brown  of  the  National  Assessment  of 
Educational  Progress,   Beth  Brenneman  of  the  California 
State  Department  of   Education's  Measurement  Division,  Jean 
Jensen  of  Las  Lomas   High  School,  Walnut  Creek,  California, 
Kate   Blickhahn  of   Sir  Francis  Drake  High  School,  Mill 
Valley  among  others.     Then  she  met  Allen  Seder  at   the  Educa- 
tional Testing  Service  office   in  Berkeley,   who  invited  her 
to  read  for   the  California  High  School  Proficiency  Examina- 
tion.    Later,   under   the  auspices  of   this  BAWP  Writing  Assess- 
ment  Project,   Ms.   Keech  spent  a  day  with  Gertrude  Conlan  and 
her   staff  at   the  Educational  Testing  Service  office  in 
Princeton  learning  about   the  procedures  used   there  for 
developing  and  piloting  writing  prompts.     From  that  meeting 
Ms.   Keech  brought  back  the  Conlan  checklist   for  writing 
prompts  that   is  appended   to  Part   II  of   this   report.  During 
her  work  for   the  Scriven  Evaluation  Unit,   Ms.   Keech  also 
became  acquainted  with  Edward  White,   the  chief  designer  of 
writing  examinations   for   admissions  to  the  California  State 
University   system,    and  with  Paul  Diederich.     During  these 
years,  Ms'.   Keech  collected  an  extensive  file  of   topics  and 
developed  an  unusual  depth  of  background  knowledge  of  current 
practices  in  designing  writing  assessment  prompts.     Thus  she 
came   to  this   project  prepared  to  write  the  draft  of  a  document 
called,   in  its  first  version,   "Topic  Design:     Views  from  the 
Profession. " 

The  Review  Panels.      In  the   late  fall  of   1980,  we  sent  a 
draft   form  of   Catharine  Keech's  manuscript  to  two  panels  of 
reviewers.     As  project  director,   I  wrote  a  cover  letter  asking 
each  person   to  serve  as  a  Critic-Reader  of  the  paper  provided. 
The   letter  mentioned  that  we  desired  to  prepare  a  technical 
report  on  topic  development  from  the  point  of  view  of  teachers, 
researchers,   and  professional  evaluators.      (The  people  re- 
sponding are   identified   in  the   report  of  the   findings  that 
follows  this  section.)     The  letter  said,   in  part,   "The  report 
will  "accumulate  illustrative   'good'   and   'poor'   topics  along 
with   sage  advice  on  the  problems  of   topic  development  and  how 
to  avoid  them."     We   invited  the  readers  to  comment  specifi- 
cally on  the  "completeness,   accuracy,   organization,   and  style 
of  the  presentation"  and   to  share  with  us  any  illuminating 
anecdotes   from  their  own  experience  relating   to  "unexpected 
behavior  of   certain   topics."     We  mentioned   that  we  expected 
to  incorporate  into   the  final  version  of  the   report  a 
summary  of   the  comments  received.     That  summary  appears  in 
the  next  section  of   this  chapter. 

The  panel  members  had  been  selected  on  the  basis  of 
their  many  years  of   experience  as  readers  in  scoring  sessions 
for  holistic  writing  assessments,   as  developers  of  writing 
prompts  themselves   in  school,    state,   and  national  assessments. 
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or  as  researchers   interested   in  writing  assessment.  Among 
the  panels  were  representatives   from  the  Educational  Testing 
Service,  which  pioneered  the  development   of   holistic  assess- 
ment  procedures,   as  well  as   several   teachers  who  pioneered 
in  adapting  these  procedures   for  use   at   local   school  or 
district  levels. 

A  Recursive,  Iterative  Model:  Theory,  Research,  and  Practice 
Interactine 


Roger   Shuy,   as  mentioned  above,   noted  the  passing  from 
favor   of   the   old  linear  model  of  knowledge  generation  (theory- 
to-research-to-practice)    in  preference   for   a  new  recursive 
model   of   theory,   research,    and  practice   interacting.  Though 
we  make  no   claims   for   the  perfection   of   our   procedure,  we 
think  we  have   conducted  a  modest   trial   of   this   iterative  model 
of   a  procedure   for   consensual  validation  of    that  professional 
wisdom  which   constitutes   a  "theory   of    idealized  practice." 

We  created  a  medium  for   collection,   exposure,    and  trial 
of   ideas--the   ideas   and  knowledge  that   arise   from  practical 
experience.      Practical  knowledge  does  not   carry   the  authority 
of   the  processed  knowledge  of   science   in  many  quarters  where 
naive   interpretations   and  applications   of   "scientific  method" 
to  educational  problems  haje  led  to  an  undervaluing   of  personal 
and  professional  knowledge.     Personal  knowledge   is  less 
orderly,   less  processed,   less   capable   of   generalization  than 
the  processed  knowledge  of   science.      But  practical  knowledge 
is  rich   in  content;    it   is   tied   to  specific   contexts;    it  has 
been  tested  under   fire.     It   is  valuable   for   it  has  withstood 
tests   of   situational  verification.      The  problem  is   that  such 
knowledge  remains   in   limited,   private  realms.      It   does  not 
circulate,   and  therefore   it   does  not  profit  by  criticism  from 
within   the   community   it  might   serve.     And   this   step   is  crucial. 

This  is  why  we   submitted   the  original  Keech  account  of 
practice  to   a  circle   of   collegial  critics.      For  when  personal 
practical  knowledge   comes  under   the   scrutiny   of   fellow  profes- 
sionals,  that  particularized  body  of   ideas,  experiences, 
hypotheses,    theories,   and  practices   comes   into   contention  with 
the   experiences  of   others   in  the  community   of  practitioners. 
The   limits   of   the  validity   of   the  ideas  proposed  are  set 
through  exchanges   directed   toward  mutual   learning.  Though 
the   community  of   professionals   shares   interests  and  commitments 
in  problems   of   the  measurement   of  writing,    the  process  preserves 
the   acceptability   of   disagreement    (as   is  noted  in  the  following 
report   of   results).      Because  different   people  have  different 
ways   of   looking  at   and  Judging   the  processes   of  conducting 
writing  assessment,   a  collective   image   of   the   reality  of  the 
process   takes   on  a  variety   of   perspectives  more  comprehensive 
than   any   individual   contribution.      Stenhouse    (1980),    in  quite 
different   language,   makes  many   of   these   same   points   in  answering 
the  question:      "What   Counts  as  Research?"    (pp.    108-112).  He 
ends  his  quest  with   the  observation:      "Research  is  educational 
to  the   extent   that   it   can  be  related   to   the   practice  of 
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education''    (p.  113). 


The  next   section  reports   the   collaborative  results  of 
our  efforts    to  refine   our   current   knowledge   about  procedures 
for  designing  writing  test  prompts. 
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RESULTS   OF  THE  STUDY 


The  open-ended   invitation   to   read  and   review  Catharine 
i^eech's  preliminary  version  of   **Practictes   in  Designing  Writing 
Test  Prompts"   elicited  responses   in  a  variety  of   forms:  textual 
editing  changes,   marginal   comments  of   varying  length,   and  cover 
letters  usually  ranging  from  one   to   three  pages^     The  frequency 
of  marginal   comments  ranged   from  eight   to  sixty  among  those 
respondent s   choosing   this  mode  of  reaction.      In  one  special 
case,    the   respondent   provided   twelve  pages   of   comments,  each 
enumerated  and  keyed   to  specific  passages   in  the  Keech  text. 
The   respondents  offered  general  comments  about  the  report  as  a 
whole,   reactions   to  specific  points  raised   in  the  report, 
relevant  anecdotal   information   from  personal  experience  in 
teaching  and  assessing  writing,   and  various   helpful  suggestions 
for  additions  and  changes  that   in  many  cases  were  incorporated 
into   the   final  version  appearing  as  Part  II   of  this   study.  Most 
of   the  comments   dealt  with  the   idea  content   of   the  report, 
although  a   few  dealt  with  format,   organization,   style.  Many 
of   the  substantive  comments  reflected   the  respondent's  own 
experience  with   the   same  or   similar  prompts. 

The  data  provided   through   this   form  of   panel   review   do  not 
yield   to  quantitative   treatment,   yet   identifiable   themes  and 
consensua  lly  validated  recommendations   for   practice   do  emerge 
through  the   collation  of  responses   from  the   community  of 
professionals  represented   in  this  sample.     The  remainder   of  this 
report   identifies   the  main  themes   and  recommendations  of  the 
reviewers   and  provides   illustrative  quotations  from  their 
submissions.     The   findings  are  presented  in   two  main  sections: 
the   first   summarizes   the  responses  of   the  writing  instructors; 
the  second   summarizes   the  responses  of   the  researcher /evaluator s 

Responses   from  the  Panel  of  Writing  Instructors 

THE  PANEL  OF  WRITING  INSTRUCTORS 


*1,     Ruby  Bernstein,   English  teacher,   Northgate  High 
School,   Concord,  California 

*2.     Mary  Frances   Claggett,   English  department  chair- 
man,  Alameda  High  School,   Alameda,  California 

3.  Kim  Davis,    Subject  A,   University  of  California, 
Berkeley,  California 

4.  Edmund     Farrell,   Professor  of   English  Education, 
Department  of   Curriculum,   University  of  Texas, 
Aus  tin 

*     Designates  Bay  Area  Writing  Project  Teacher /Consultant 

11., 


*5.     Jean  Jenson,   English  department  chairman. 

Las   Lomas  High  School,   Walnut   Creek,  California 

*6.     Irving  Peckham,   English  teacher.   Live  Oak  High 
School,   Morgan  Hill,  California 

7.     Bill  Robinson,  Director  of   Composition,  English 
Department,   San  Francisco  State  University 

*8.     Bernard  Tanner,   English  teacher,   Palo  Alto  High 
School,   Palo  Alto,  California 

Value  of   the   Study.     The  eight  writing  instructors  who 
evaluated  the 'trial  version  of   the  report  were  unanimous  in 
their  praise  of   the  document   in  general.     They  all  valued  the 
attempt   to   compile  information  about   the  design  of  writing 
prompts  and  assured   the  researchers  of   the  need  for  the 
guidance  it   aimed  .to  offer.     Ruby  Bernstein,   a  high  school 
teacher,   thought   that   such  a  study  would  be  especially 
significant   for   teachers  of  English  and  that   it   could  be 
useful,   as  well,   to   teachers  of   subjects  across  the  curriculum. 
She  felt  the  work  gave  her  insight   into  how  "boring  papers" 
produced  by  students  might  result  from  the  way  she  had  framed 
the   initial   topic,   and  she  suggested  workshops  be  conducted  for 
teachers  in   the  development  of  writing  prompts. 

Bill  Robinson,   Director   of   Composition  at   San  Francisco 
State  University,   called  the  report   a  "very  good  Job,  covering 
in  an  intelligent   and  helpful  way  a  really   tough  problem. 
Kim  Davis,  Director  of  Subject  A  at   the  University  of  California, 
Berkeley,   said  "This   study  should  become  required  reading  for 
all  English  teachers--a  great  many  of  whom  give  little  hard 
thought,   I   suspect,    to  the  writing  tasks  they  continually  set 

for  students  "     And  Edmund  Farrell,   Professor  of  English 

Education  at   the  University  of  Texas  at  Austin,   though  praising 
the  efforts,   suggested  that   the  study  should  address  itself 
more  "to  classroom  teachers,  who  flounder   in  their  attempts  to 
create  decent   assignments."     However,   Farrell  feared  that  the 
study  won't   get  "the   circulation  it   deserves,"  and  suggested 
revising  it   for   submission  to  NCTE/ERIC  as  a  TRIP  booklet.  Kim 
Davis  also  mentioned  widening   the  scope  of   the  study  to  include 
creation  of  writing  assignments  in  general — i.e.,  "routine 
classroom  assignment s"--not  just  for   assessment  purposes. 

Classification  of  Writing  Prompts  bv  Discourse  Mode.  Most 
of  the  reviewers  were  enthusiastic  about   the  draft  of   the  report 
they  received.     They  recognized  it  as  a  work  in  progress  with 
great  potential,   and   they  welcomed   this  opportunity  to  offer 
constructive   criticisms  and   to  contribute   to  its  fuller 
development.     One  feature  of   the  report  most   in  need  of  recon- 
sideration,  according   to  several  of   the  reviewers,  was  the 
scheme  adopted   for   classifying  types  of  writing  prompts.  This 
scheme,   based  on  James  Britton's  discourse  model,   sorted  prompts 
into  three  broad  categories:     transactional,  expressive,  and 
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poetic.      Both  Bernard   Tanner   and   Bill  Robinson   considered  the 
weakness  of  this   classification  in  some  detail- 
Bernard  Tanner,   high  school  English   teacher,   stated  that 
he  did  not   "buy  Britton's   intellectualized  categorizing  of 

youngsters'   writings  "     Tanner   challenged  Britton  s 

functional   classification  of  writing,   explaining   that  he  could 
never   be   sure  whether   Britton's   categories  derived   from  the 
•natural'    modes   adopted  by   children  or  whether   they  are  the 
result,   merely,   of   the  kinds   of   tasks  British     ""^ers  call 
for...."     Tanner   spoke   of   "the  pinch  of   categorical  definition 
that   overlooked   too  much.      He  also   argued   that   "no  categories 
prove   exclusive   of  aspects   of   others,"   and  he   cited  many 
examples  of   dubious   classifications    (in  his  view)   to  prove 
his  point.      Tanner  himself   finds   the  older  rhetorical  labels 
more   helpful,    and  as   he  prepares   a  writing  prompt,   he  asks 
himself   the   question:      "What  mix  of  modes   of   discourse   do  I 
wish   to  lead   the  writer  toward?" 

Another   of   the  writing   instructors,    speaking   from  his 
vantage  point   at   the   college   level,   also   considered   the  limita- 
tions of   the   classification   scheme  presented.      Bill  Robinson 
presented   a  strong  argument   against  using  Britton's  functional 
triad,    transactional   -  expressive   -  poetic: 

I   am  not  persuaded  that   the  modern  approaches   to  ^ 
mode   are   any  more  helpful   than  Bain's,   and  Britton  s 
are   in   fact  heavily  value-laden.      Sending  teachers 
to  him,   or  worse,   Emig   for   guidance   is,    I   think,  a 
mistake.      In  addition,    the   fact   that  you  are 
addressing   this   paper   to  teachers   or  evaluators  of 
children  of  wildly  different   ages  and  abilities 
seriously   complicates   your  problem.      Essential  to 
Britton's   categories   are   the  various   stances  the 
writer   assumes  or   is   forced   to  assume  vls-a-yis - the 
audience.      In  his  "expressive"  mode,   the   audience  is 
nonjudgmental  to   the  point   of  being  paternal  or 
matirnal.      It  seems   to  me  highly  unlikely   that  such 
an  atmosphere  can  be  set  up   in  a  testing  situation 
with  any  but  the  teeny  tiniest  of   little  kids  or  the 
dopiest.      In  addition,   whether   the  mode   is  expressive 
or   transactional,   one   still  has   to  decide  what  form 
of  writing  one  wishes   to  evaluate--or   let   us  put  It 
slightly  differently-what   skill  one  wishes  the  students 
to   demonstrate.      Does  one  wish  to   see  whether   they  can 
tell   a  story  of   their   own  devising?      (Note,  however, 
that  willy  nilly  one  gets   into  testing  creativity  at 
this  point.)     Does  one  wish   to  see  whether  they  "n 
recount  an  anecdote   from  their  exper ience--l . e . ,  handle 
simple   narrations?     Does  one  wish   to  see  whether  they 
can  discuss   and   evaluate  an   idea   of   some   sort   In  terms 
of   thei.   own  experience-i.e.,   handle  «i"Pl%"P°^'^J°" ' 
Does   one  wish  to  see  whether   they   can  evaluate   the  pros 
and  cons  of  a  problem  with  which  they  can  be  expected 


100 


to  be  f amiliar--i. e . ,   argue  validly?     I   think  it 
would  be  much  more  helpful  to   lay  out   the  potential 
tasks  to  be  tested  in  ways  such  as   this  rather  than 
to  examine  some  extremely  dubious  categories. 

On  the  basis  of   these  recommendations  and  others  made  by  the 
researcher /evaluators ,   the   final  version  of   the  report  does 
not   attempt  to  apply   the  Britton  scheme  to  the  classification 
of  prompts   for  writing  assessments. 

The  Content  and  Wording  of  Prompts.     The  teacher-reviewers 
offered  many  comments  about    the  content  and  wording  of  the 
sample  prompts  that  had  been  included  in  the   trial  version  of 
the  report.     These  reviewers  responded  to  specific  prompts  as 
a  wholet    to  portions  of  them,  or  even  to  single  words  in 
individual  prompts  on  the  basis  of  their  own  experience  of 
having  had  students  write  to  the  same  or  similar  prompts.  Some 
described  positive  experiences;   others  reported  problems. 
Some  suggested   changes  in  the  wording  of  the  topic  or   in  the 
instructions.     Others  recommended  abandonment  of  particular 
types.     Representative  comments  follow. 

Ruby  Bernstein,   a  high  school  English  teacher,  pointed 
out   that  ETS  argument   topics  or  prompts  don't   elicit  good 
writing,   that   students  have  a  "livelier  voice,  have  more  to 
say  when  they  are  allowed   to  recreate  experience."  Jean 
Jensen  reported   that  her  ninth  and   tenth  grade  students 
responded  very  well  to  "story  starters."     Jensen  also  observed 
that   certain  prompts   can  be  expected  to  elicit  different 
levels  of  performance  depending  on  grade  level,   ability  level , 
and   community    (i.e.    suburbia  vs.    inner  city).     Irving  Peckham, 
a  high  school  Engl  isfb -teacher ,     also  referred  to  the  effect 
of  age  or   stage  of   the  student  on  response  to  certain  prompts, 
but  he  observed  that  making  up  a  story  is  an  easier   task  for 
some   individual  students,   regardless  of  age  group  or-  "stage 
of   development,"  as  posited   in  the  study.     Tanner  commented 
that   students   tend  to  carry  over  their  own  patterns  of  speech 
and   thinking  into  first  person  stories.     He  concluded  that 
"they  must  be  prompted  to  invent  a  character  or  a  point  of 
view  different   from  their  own  natural  propensities."  Tanner 
said   that   teenagers  will  write  stories,   but  "they  tend  always 
to  use  themselves  as  the  central  figure... if  you  give  them  a 
specific  role  which  takes  them  out  of   themselves,   they  often 
flounder  or  write  only  briefly."     Tanner  also  addressed  the 
need   for   specification  of  plot: 

Unless  the  prompt   includes  a  clearly  defined  initial 
incident  which  has   the  potential,    .ight   in  the  prompt 
itself,  of  being  a  problem  to  resolve,   one  is  unlikely 
to  find  many  students,   even  students  of  creative 
writing,  who  can  develop  a  plot.     Students  can  develop 
a  chain  of   happenings,   but   that   is  not  necessarily  a 
plot. 
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With  reference   to   the   content   of   the  prompts,   Kim  Davis 
reported  on   certain  prompts   that   Subject  A  has   learned  to 
avoid:    education,   writing,    popular   controversial  issues, 
humor.      On  the   first   two  of   these,   students  write  what  they 
assume  readers  want   to  read;   the   third  ''produces  a  depressing 
blather  of   cliches;"   and  the  fourth  is   ''a  subject   so  far 
removed   from  the   circumstances  of   a  writing  exam  that  students 
are  rarely  amused  by   it."     But   the  Subject  A  faculty  do  want 
students   to  write  argumentative   expositions,    although  they 
elicit   it  without   resorting  to  "the   dreary   likes  of  'abortion/ 
'capital   punishment,'    'marijuana,'   etc."     In   the  matter  of 
political   topics  or  prompts.   Bill  Robinson  warned   that  the 
wording  of   a  prompt   should  not   force  students   to  write  from 
a  bias   they  don't   share.     He  advised   caution,  "remembering 
always   that   a   school   topic   is  not   context   free."  Because 
students   always  know  who  will  be   reading  their  papers, 
Robinson  said,    using  topics  that    elicit   strong  opinions 
could  be  "very  touchy." 

Tanner  brought   up  an   issue   that  no  other  reviewer 
considered:   the  role   of  knowledge   in  affecting  the  writer's 
point   of  view  or  stance,   purpose,   and  vocabulary.      As  he  put 
it,   "writing  is   subject   specific."     A  person  must   have  some- 
thing in  mind   to  write   about,   some  knowledge   of  a  subject. 
And  how  that   subject    is   developed  affects  the  way  the  reader 
judges   its  merit.      Tanner  provided  several   illustrations  of 
his  point : 

One  can  write   about   history   only  when  he  has  learned 
some  history  and   learned  a  bit   about  how  history  can 
be  written.     The   subject  makes  demands   on  vocabulary. 
It  makes   demands   on   the  writer's   stance.      It  makes 
demands  on  what    facts   should  be   selected  and  how  they 
should  be   put   in  order ....  Some  people  may  want   to  use 
writing  prompts   to   find  out    if   a   student   can  write  about 
chemistry,   or  metal-working. 


Tanner  con t  inued : 


...when  we  holistic   readers   respond  this  way  we  are 
responding  to   subject  matter  sense  rather  than  to  mere 
writing  skills.      The   irony   is  that   one   can  write  error- 
free  prose  about   nonsense.      When  we  read  AP  papers  for 
ETS  we  are  reading  holistically   for  BOTH  content:  and 
quality  of   expression.     We   are  facing   the  SUBJECT 
SPECIFIC  demands   on   any  worthwhile  writing.... 

....What    I  am   saying   is  this:  No    'transactional*  prompt 
can  be  truly  broa*dly   successful  without   depending  on 
reading  skills   or  without   being   intent  ionally_t he 
culminating  act   to   some  body  of   study   or  learning 
experience .      Hence   in   the  nature  of  things  one  must 
evaluate   such  writing  on  a  basis   of  subject  matter  as 
well   as  writing   skil 1 s . . . . Fr om  this  monograph  one  would 
think  that  man   can   genuinely  write   in   a  vacuum  of 
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reference,   or  at   least  that  the  reader  can  read  a 
response  without   giving  a  hang  whether  or  not   it  means 
anything . 

Tanner  also  offered  trenchant   comments  on  the  wording  of 
prompts.     He  cited  a  case  where  he  was  question  leader  at  the 
reading  of  an  Advanced  Placement   English  test    (in  about  1971 
or  1972)   which  used  a  question  based  on  a  passage  from  an 
Orwell  essay  as  a  springboard: 

The  instruction  went   something  like  this:  "Demonstrate 
how  the   author  of  this  passage  establishes  his  attitude 
toward  the  coming  of  spring."     I  was  question  leader  on 
that   question,   and  early  on   in  the  reading  I  was  puzzled 
to  have  large  numbers  of  papers  brought   to  me  with  that 
question  blank.     Many  obviously  fine  candidates,  who  did 
fine  essays  on  the  other  two  questions,   simply  skipped 
that   first   question  entirely.     And  then  I  had  a  notion: 
I  suspect   that   a  great  many   students  took  that  word 
"Demonstrate"  to  mean  that   they  should  pick  subjects  of 
their  own  and  imitate  Orwell's  manner  of  expression.  I 
was  led   to  this  belief  by  one  paper  in  which  a  good 
candidate  had  actually  done  that   as  he  'demonstrated 
his  own   attitude   toward   the  beach. 

Tanner  then  gave  an  account  of  how  he  writes  prompts  to  avoid 
such  wording  confusions: 

I  have  since  made   it   a  rule  of  thumb  for  myself  in 
attempting  to  write  prompts  that   I  would  use  an  entirely 
limited  vocabulary.      I  use   "tell"  when  I  mean  either 
narration  or  description.      "Tell  what  happened... 
means  narration;   "Tell  what  the  place   looks  like... 
means   description.     I  use  "explain"  when  I  mean  either 
persuasion  or  exposition.      "Explain  how  it  works... 
means  exposition;   "Explain  what   you  believe  and  why... 
looks  toward  persuasion.      (Naturally  I   do  not  mean  that 
I  use  those  quotations;   I  mean  merely  that  by  using 
"tell"   and  "explain"  and  whatever  appropriate  phrases 
are  needed  after  these  commands,   I  establish  the  central 
mode  or   intention  of  the  piece  of  called-for  writing.) 

Tanner   illustrated  how  the   substitution  of  "tell"  for 
"describe"   clarifies   the  meaning  of  a  particular  prompt, 
which  could  be  interpreted  to  mean  "describe   an  object  which 
taught  you  a  lesson."     Tanner  commented: 

A  student  might   scratch  his  brains  for  some  time  to 
invent   a  "lesson"  taught   him  by   some  particular  object— 
a  hard  sidewalk  or  something— an d  end  up  with  a  rather 
silly  piece  of  trumped-up  writing.     Try  "Tell  about  an 
experience  from  which  you  learned  a  lesson.      lorj  ieii 
about   an  activity  you  enjoyed  doing  as  a  child. 
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Purposes  of  Writing  Assessment   and  Rhetorical  Purpose. 
The  issue  of  purpose  was  discussed  in  two  contexts:  Several 
teachers  agreed  on   the   Importance  of  writing  with  a  rhetorical 
purpose   in  mind--l.e.,   that   the  students  should  know  what  aim 
their  writing   Is   to   accomplish.      This  notion   also   relates  to 
the   concept    of  audience.      But   several   others--lncludlng  some 
of  the  researchers   and  professional  evaluat ors--addresse d  the 
concept   of  "purpose"   In  terms  of  the  tester's  reasons  for 
having   students  write   to   these  prompts,   remarking   that  this 
objective  hadn't   been  made   clear    In   this  preliminary  version 
of  the  report.     Tanner  said: 

Right   now,   I   find   only   the  vaguest   things   said  about 
various   purposes   for   testing   of  writing.      If   the  paper 
is   going   to  mention   "purposes"   shouldn't   it   show  a  list 
of   purposes,    the  possible  range   of  purposes?      Shouldn  t 
these   then  be   discussed   in   relation   to   possible   types  of 
prompts  which   can  be   related  to   these   purposes?     I  would 
try   to   discuss  ways   in   which   the  wording  of   a  prompt 
relates   to  any   attempt   at   holistic  reading. 

Tanner   seems   to   be  asking  about   the   function   of  the 
writing  assessment   presented   in   the   trial  version   of  the 
report.      Is   it   Intended  to   assess   writing   skills   alone,  or 
is   it    intended  to   assess  writing   skills  as   they   can  be  brought 
to  bear   on   the   communication   of   some   sort   of   sub  j  ect-specl  f  ir. 
content  ? 

With   regard   to   purpose   Insofar   as  it    Includes  audience, 
most   teacher-critics   agreed   that    it    is  useful  to   specify  an 
audience-    however,    Peckham  warned  that   students   find  certain 
instructions   about   audience   to  be   "phony:"     "This   is  the 
problem   I've   felt   with  the   audience   craze.      Quite   often  a 
phony   audience   is   as  bad   as  no   audience  at   all.      Clear  purpose 
seems  more   important.      For   expressive  and   poetic  writing,  no 
real  audience   is  necessary." 

And   finally,    with  regard  to   the   prompts,   Kim  Davis  asked 
about   the   "reality"   of   students   having  to  write   to  topics 
they'd  just    as   soon   not   write   to:      "How  should   students  be 
counseled   to  make   the  best    of   a  bad   situation?"     After  all, 
no  matter   how  carefully   the   purpose   for  the  writing  and  the 
audience   for   the  writing   are   specified,   students  writing  In^ 
a   timed   assessment    situation   are  no_t   going   to  produce  real 
writing,   no  matter  what   the   instructions   read.      What   is  being 
assessed,   then,    is  not   the  students'   ability  to  write  in 
"real"   situations,    but   the   students'    ability   to   perform  in  a 
highly   artificially   structured   "test"  situation,  InN^hlch 
they  agree  to  play  the  role   of   "real  writers,"  and  the 
teacher-reader   agrees   to   play   the   role  of   a   "real  reader. 
In    fact,    they   are   really   acting   in   their   ordinary   roles  ot 
student-performer   and  t ea ch er-e v alu at  or . 

The  Function   of   Instructions.      Davis   also   agreed  that 
the  writing   instructions   are   Import ant--t hat   It   Is  the 

104  -  • 


ERIC 


instructions  rather   than  the   topic  that   determine   the  purpose 
of   the  writing.     Peckham  agreed   that   Instructions   for  writing 
are  best   left   simple.      "Overwriting  indicates  uncertainty 
about   the   topic/'  he   said.      Jensen  agreed   fully  with  the  need 
to  avoid  overprompt ing   and  underprompt ing  in   giving  instruc- 
tions.    Tanner's  anecdote  about   the  misunderstanding  of  the 
term  "demonstrate,"  mentioned  above,   illustrates   the  fact 
that   sometimes  a  single  word   in   the   instructions   can  mislead 
the  writer. 

Two  areas  under   the  "Use   and  Misuse  of  Constraints" 
section  of   the  trial  version  were  discussed  by  several  of 
the  teacher-critics:   Under  the  "Pick-A-Form  Trap."  Bernstein 
noted  that   the  letter   form  has   special  problems.     The  "form" 
part   takes   up  too  much  time  and   space,   and,   when  used   in  her 
school,   the   results  were  boring.      She  called   this   an  example 
of   "testing   for  the  wrong  thing  or  putting  the  parts  before 
the   content."     Robinson  asserted  that   there  were  additional 
problems  with  asking   students   to  pick  a  form,   since   this  task 
of   choosing  a   form  added  to  the  work  the   student  was  required 
to  perform   in  a  limited  amount   of   time.     Also,   a  given  student 
may  not  pick  the  form   that  would  have  produced  the  best 
results   for   that  particular   topic.      And  most   important  for 
purposes  of  assessment,   readers   or  evaluators  have  problems 
evaluating   the  same   topic  written   in   different  forms. 
Robinson  said,   ''I  do  not  believe  you  can  have  a  valid  test 
situation   in  which  students  may  pick  different   forms  for 
their  responses."     (Several  of   the   researchers  and  profes- 
sional evaluators  also  questioned  allowing  the  students  to 
"pick  a  form"   on  a  writing  assessment  task.) 

Most   of   the  teacher-critics  agreed   fully  that  specifying 
number  of  words,   sentences,    or  paragraphs  was  not   only  an 
unhelpful   constraint,   but   it   adversely  affected  the  writing 
that  was  produced.     Peckham  said  of   instructions  specifying 
number  of  words:   "They  encourage  a  minimum  response."  And 
Bernstein  pointed  out   that   the  ETS.test  booklet   itself  is  a 
"constraint,"  since   it   provides   the  student  with  a  limited 
and   specified  number  of   lines   to   fill  up   in  20  minutes. 

Time   Constraints.     The   fact   that   these  prompts  were  being 
developed   for  use   in   timed  assessments  was  questioned  by  three 
of   the  teacher-critics:   Farrell   stated  that   the  study  should 
not  be  limited  to  use  with  timed  tests.      Irving  Peckham 
offered  a   special  plea   for  enough  time  to  rewrite:   "We  allow 
two  periods  at  Live  Oak.     As  well  as  stimulating  rewriting 
in   the  classroom,   our   insistence  on  second  drafts   is  a 
blessing   for  readers." 

Bill   Robinson  also  expressed   concern  about   the  effects 
of  time  pressure  on   the  type   of  writing  produced: 

...the  time  allotted   for  response  to  a  question  has 
nothing  to  do  with  the  mode  of  response   to  be  elicited. 
That   is,   a  speeded  test  will  not  promote  expressive 
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writings   at   least  not  in  Britton's  definition   of  expres- 
sive.    What   a  speeded  test  will  produce   is  writing  more 
heavily  characterized  by  oral-composing   traits  than  a 
relatively  non-speeded  test.     That   is,   the  heavier  the 
cognitive   demands  produced  by  the  combination  of 
response  desired   and  time  given,   the  more  the  response 
will  resemble  spontaneous  speech,   with  all  its  ''errors" 
and  ungrammat icalit ies ....the   factor  of   time  is  one  you 
haven't   discussed  as  such,   and  yet   it   is,   of  course, 
very  important. 

Use  of  Pilot  Tests.     Two  of  the  t ea cher- c r it i c s  agreed 
with  the  need  to  pilot   test  the  prompts  before  using  them  in 
a   full   assessment;   however,   both  also   stressed  the  importance 
of  having  teachers  write  to  the  prompts  as  well.     Jean  Jensen, 
a  high  school  teacher,    reported  her  experience  at   one  school 
with  pilot   testing  prompts  with  students:   Despite  their  pilot 
testing,   they  still  often  received  boring  papers.     At  another 
school,   where  pilot  testing  was  done  among  the  teachers 
rather   than  among  students,   she  reported  that   the  student 
papers   were  better  than  at   the  other   school,   but   she  and  her 
colleagues  were  not  able  to  suggest   reasons   for  this.  Jensen 
stated,         think  it   is  tremendously  important   that  these 
teacher /test-makers  write  on  the  prompt   themselves   and  talk 
about   the  effect   of  a   question  on  them.. ..The  teacher  writing 
himself   is  crucial ! . . .When  teachers  themselves   also   test  the 
question — little  trouble!" 

Responses  from  the  Panel  of  Researchers 
and  Professional  Evaluators 

Ten  researchers   and  professional   evaluators  submitted 
written   reviews  of  Catharine  Keech's  trial  version  of 
"Practices  in  Designing  Writing  Test   Prompts."     These  reviews 
consisted  of   1-  to  3-page  letters  giving  general  reactions  to 
the  whole  report.     Half  of  these  respondents   also  provided 
from  10  to  30  textual   annotations  on  the  returned  manuscript. 
As  with   the  teacher-reviewers,   most   of   the  re  searcher /evalu- 
ator  comments   dealt   with  the   content   of  the   report,  although 
some  commented  on  the   format,   organization,   and  style. 
Though   there  are  areas  of  common  interest,   the   responses  of 
the  two   review  groups   are  not   precisely  parallel.      The  main 
themes  and  recommendations  in  these  responses   are  collated 
below. 

THE  PANEL  OF  RESEARCHERS   AND  PROFESSIONAL  EVALUATORS 

1.  Evans  Alloway,   Director  of  Test  Development, 
Educational  Test  in g  Service 

2.  Gordon  Brossell,  Associate  Professor   of  English,  The 
Florida  State  University ,  Tallahassee 

3.  Rexford  Brown,   Edifor  of  Publications  and  Test 
Developer,  National  Assessment  of  Educational  Progress 


4.  Charles  R.  Cooper,  Coordinator  of  Writing  Programs, 
Department  of  Literature,  University  of  California, 
San  Diego 

5.  Paul  Diederich,   Senior  Research  Analyst,  Emeritus, 
Educational  Testing  Service 

6.  Sarah  W.  Freedman,  Assistant  Professor  of  Education, 
Language  and  Literacy  Division,  School  of  Education, 
University  of   California,  Berkeley 

7.  Ann  Humes,   Communication   Skills  Project  Assistant, 
Southwest  Regional  Laboratory 

8.  Ina  Mullis,    Senior   Research  Analyst,   National  Assess- 
ment  of  Educational  Progress 

9.  Joe  M.    Steele,   Research  Psychologist,   College  Outcome 
Measures  Project,   The  American  College  Testing 
Program 

10.      Edward  White,   English  Test   Center,   Department  of 

English,   California   State  University,   San  Bernardino 

Value   of   the  Study,      The  panel  of   researcher /evaluators 
generally  praised  the   study   as  being  greatly  needed   and  for 
including  many  important   and  useful  ideas.      They  considered 
it   both  helpful  and   interesting.     Though  several  acknowledged 
the  draft   form,   they  also,   nevertheless,   pointed  out   the  need 
for  a   careful   revision,    including  a  tightening  of   the  style 
and  organization.      Charles   Cooper,   a  writing  researcher  at 
the  University  of  California,   San  Diego,   supported   the  need 
for  this  study  and  applauded   its  achievement,   but   he  con- 
sidered  the   report   to  be  too   conjectural  in  some  respects 
and  indicated  a  need   of  more   research  to  resolve  some  of  the 
issues   it  raised. 

Issues   of  Assessment   Purpose.     The   interaction  between 
assessment   purpose  and  the  nature  of  writing  prompts  to 
accomplish  particular   assessment   aims  was  an   issue  with 
several   of   the  respondents.      Ina  Mullis,   Senior  Research 
Analyst    for   the  National  Assessment  of   Educational  Progress, 
said  that   clarification  of   the  context,   purpose,   and  audience 
for  the   report   was  needed.      In  reading  the   report,  Mullis 
could  not  be   sure  what   purpose  tne  topics  were   intended  to 
serve:    instruction  or  assessment.      If  the  purpose  was  assess- 
ment,   then  the  purpose  of   the  assessment  was  not   clear.  What 
kinds   of  decisions  would  a  tester  be  making  on  the   basis  of 
the  results?     As  indicated   in  the  "Sources  of  Knowledge" 
section  of   this   report,   Mullis   follows  NAEP ' s   "primary  trait" 
system  which  calls   for  narrowly   delineated  writing  exercises 
to  assess  particular   accomplishments  in  writing  according  to 
rubrics   established  prior   to   testing.     The  approach   in  the 
Keech  manuscript   followed  a  much  more  open  procedure  amenable 
to  holistic   scoring  and   ranking  of   the  quality  of  papers 
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according  to   rubrics   developed  during  scoring. 


Several   of  the   researcher/professional  evaluators, 
although  supporting   the   study,    raised   questions  about  the 
purpose  of   the   assessment   procedure  described  in  it.  Gordon 
Brossell,   a  writing   researcher  at   Florida   State  University, 
suggested  that   test   makers  need  to   determine  what  intellec- 
tual  operations  are   to  be   assessed  in  addition  to  writing 
before  they  begin  to   construct   a  test.     His  implication-- 
that  it  is  not   possible  to   test   writing  skills   isolated  from 
other   skills--echoed   Bernard  Tanner's   statements  about 
subject-specific  writing.      Just   as  one  of   the  pitfalls  of 
writing  prompts  is   to   avoid  making  them  into   reading  tests, 
so  Brossell   suggested  that   test   makers  det ermine  what  intel- 
lect ual  operations   are  to   be   assessed--in  addition  to 
writ ing--bef ore  they  begin   to   construct   a   test.  Sarah 
Freedman,   a  writing  researcher,    agreed  that   it  was   good  to 
stress   the  importance  of   deciding  what   one  wants   to  test 
before   designing  the  task.      Joe   Steele,   Research  Psychologist 
for   the  American  College  Testing  Program,    asked   for   a  clearer 
distinction   to  be  made  between  writing  instruction  and 
writing  assessment:    "...evaluation  of  writing  may   call  for 
a  quite   different   approach   and   set   of   considerations  than 
are  most  useful  for   instructional  purposes.''     He   said  that 
he  "found  the   attempt   to   deal  with  topics   from  elementary 
to  college   and   for  varied   instructional  and   testing  purposes 
to  be   too  big   a  task  to  carry   off  well." 

Brossell  was   unsure   "why   formal   testing  of  writing  in 
the  elementary  school  needs  to  be  done."     If   for  diagnosis 
or  evaluation,   he  asserted,    these  can   be   done  without  formal 
testing.      He   said  this   confusion  may   result   from  a  blurring 
of  the   formal   testing  of  writing  with   classroom  writing 
assignments   aimed  at   giving  a   teacher   some   evidence   of  a 
student's  ability   and  progress   in  writing.      It   is  interesting 
that   while  a  number   of  the  teacher-critics   suggested  that  the 
focus   of  the   study  be  broadened  to   include   development  of 
prompts  for   classroom  instructional  purposes  as  well  as 
assessment   purposes,   several   of   the   researchers   and  profes- 
sional evaluators  asked  for  a  clearer   distinction  between 
writing  prompts   for   the  two   purposes--of   instruction  and 
eva luat  ion . 

Need   for  Definitions.      That   there   is   not   yet  common 
agreement   among  researchers  and  professional  evaluators  on 
the   usage  of   terms   to  describe   the  various   aspects   of  writing 
assessment   is   evident   in  the  responses   of   those  who  called 
for   definitions.      Gordon  Brossell  suggested  a  source  of 
possible  conf usion--that   research  in   the   composing  process 
of  writers   should  not  be  applied  to   test   situations  unless 
such   research   involved  the   composing  processes   of  writers  in 
test    situations.      Joe  Steele  called   the   distinction  between 
literature   on   scoring  writing   samples   and   direct  ions  for 
developing  good  topics  artificial.      He   said  the  criterion 
for  judging  a  topic   "good"   is   the  scoring  procedure  itself; 
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that   the   definition  of   a  "good"   test   is    (a)   that   it  measures 
growth,    and    (b)    that    it   "must   discriminate,   and  do  so   in  a 
range  of  performance   that   is  of   interest."     Along  these  same 
lines,   Paul  Diederich,   Senior  Research  Analyst   Emeritus  for 
the  Educational   Testing  Service,   gave  the  definition   of  a 
good  topic,    in   which   the  criterion   is  not   that   it  makes 
everybody  write  well;   rather,   "it    is  one  that  yields   a  wide 
standard  deviat ion-- f rom  those  who  write  with  genius   to  those 
who  can*t  write   at   all."     Joe  Steele  suggested  that  one 
"criterion   for  Judging  that   an  essay  test   is  good  is   that  it 
measures  writ  in g  that    is  valued   as  good  outside  the  class- 
room. "     Two   of   the  professional  evaluator-crit ics  referred 
specifically  to   aspects  of   holistic  assessment   that  are 
different   from  other  kinds   of  assessment.     Rex  Brown,  Editor 
of  Publications   and  Test  Writer  for  the  National  Assessment 
of  Educational  Progress,   stated  that  many  of  the  problems 
and  traps  occur  because  the  topics  are  being  scored  holis- 
tically   rather   than  being  keyed  to   specifications  (e.g., 
primary  trait    guides),   and   the  two  kinds  are  not  inter- 
changeable.     However,   Ann  Humes,   Communication  Skills  Project 
Assistant   at    SWRL,   stated  that   "one  of   the  assets  of  holistic 
assessment   is   that   prompts   can  be   less  expl ic i t /comp 1 icate d 
because. .. [one   is]   not   assessing   specific  features." 

And  when   it   comes  to   defining  and   classifying  types  of 
writing  prompts,   as  with  the  teacher-critics,    several   in  this 
group   criticized  the   use  of   the   Britton   classification  system,. 
Gordon   Brossell   said   that   becoming  acquainted  with  a  modern 
classification   system  probably  is  irrelevant   to  the  actual 
task  of  writing  prompts.      Sarah  Freedman  said  that   the  main 
problem  was   that   Britton's   categories   don't   fit.      She  said, 
"I'd   like  your   topic    classification  scheme  to  capture  dif- 
ferent   levels   of  cognitive   demand  that   can  be  placed  on 
writers  by   different  topics." 

The  Content   and  Wording  of  Prompts.      Several  respondents 
criticized  various  aspects   of  the  prompts,   giving  reasons  and 
suggesting   improvements.     Ann  Humes  pointed  out  problems  not 
only  with  the  wording  of  some  of   the  prompts,   but   also  with 
the   fact   that    it  was  not  always   clear  what  the  intended  grade 
level   for  each  prompt   was.      In  pointing  out   weaknesses  in  the 
wording  of   certain  prompts,   Edward  White,   English  Test  Center, 
California   State  University  system,  was   critical  of   the  inclu- 
sion of   some   of   the   examples   for   fear   that  they  would  become 
models  of  good  practice   for   those  who  might  not   read  the  study 
carefully.      Regarding  types   of  prompts.   Rex  Brown  suggested 
avoiding  not   only  yes-no  questions,   but   also  "disingenuous 
pro  and  con  questions."     Ina, Mullis  agreed  that  story  starters 
or  endings   should  not   be  used  with  older  writers,   but  she 
cautioned  that   "older"  writers  are  not  necessarily  "more 
experienced"  writfers.     Working  with  national   samples,  Mullis 
reported,   NAEP   found   that    17  year-olds   generally  were  not 
"experienced"  writers.     "They  tend  to  be  grateful   for  addi- 
tional prompts,   especially   in  a  timed  situation. ||  (Note:^^ 
Mullis  probably  means  "instructions"  instead  of  "prompts," 
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in  the   context   of  this   study's  terminology.)     She  also 
cautioned  against   "opinion"  topics  with  the  interesting 
observation  that  writers  who  "agree"  have   a  headstart  on 
writers  who  "disagree,"   since  the   latter  have  to  stop  and 
take  time  to   formulate  their   thoughts--to   decide  on  what 
basis   they  disagree.      Also,   the  more  complex  the  initial 
statement,   the  more   difficult   the  task  will  be   for   the  writer 
who   doesn't   want   to  agree. 

And   finally.   Rex  Brown  objected  to  the  "trees"  prompt: 
"It  elicits   too  broad  a  range  of   responses.     For  me,  the 
worst   pool  of  papers    for  holistic   scoring   is   a  pool  con- 
taining  all   discourse  modes   and  many  very   different  response 
strategies."     He  asserted  that   a  topic   for  holistic  scoring 
should  minimize  the  number  of   dimensions   along  which  the 
papers  will  be  compared.      (This  would  also   seem  to   be  an 
argument   against   allowing  students   to  "pick  a  form.") 

Purpose   and   Audience.      Several   of   the  respondents 
discussed  the   concepts   of  purpose   and   audience   for  writing. 
Ed  White  agreed  with   the  study's   emphasis   on  the  need  for 
specifying  rhetorical   purpose.      Charles   Cooper  stated, 
"Specifying  audience   seems  crucial  to  me   in  all  explanatory 
and  persuasion  writing."     Sarah  Freedman   asserted   that  All 
test   writing   should  by   definition  be  audience-centered. 
However,   Gordon  Brossell  said   that   providing  "the  full- 
rhetor  i  cal-  c  ont  ex  t   version  of   a   test   hindered  rather  than 
helped  writers.      Creating  a  naturalistic  writing  problem 
does  not   assure  that  writers  will  respond   to  it   any  better 
than   to   another  version." 

Functions  of   Instructions.      With  regard  to  instructions, 
Brossell  noted  that   not   only   are   their   content   and  style 
important,    but   "the   tone  of   instructions  in   tests   can  do 
something  to   alleviate   test    anxiety."     Several  other  of  the 
respondents   discussed  various  aspects  of   the  section  on  Use 
and  Misuse   of  Constraints."     In   the   "Pick-a-Form  Trap 
section     echoing  Rex  Brown's   earlier   comment   as  well  as 
responses   from  several   of  the  teacher-critics,    Sarah  Freedman 
questioned   giving   students   a   choice  of   rhetorical   form  on  the 
grounds   that   this   would  not    result   in  a  good  ranked  scoring. 
In  the  "Special  Knowledge  Trap"   section,   Freedman  underscored 
the  necessity  of  providing  prompts  based  on  universal  experi- 
ences  and  allowing    for   choices   within  a   given  structure,  as 
is  the   case  with  ETS,   EPT  topics,   and  junior  proficiency 
topics   at   San  Francisco  State  University. 

With  regard   to   specifying  number   of  words,   sentences,  or 
paragraphs,   Ann  Humes,   along  with  a  number  of  others,  agreed 
that    it   is   best  not   to  have   such  constraints,   but   she  also 
noted   that   often  a   district   will  mandate   that   students  be 
able   to  write  a  certain  number  of   sentences,   and  this  kind 
of  policy,    especially  when   it   is   part   of   a  "skills  curriculum, 
gives   rise  to  this   kind  of   specification.      And  although  it  is 
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generally  agreed  that   skills  such  as   sentence  recognition, 
punctuation,    and  other  mechanics   can  be  measured  using  objec- 
tive tests,    Ina  Mullis   reported  that   ''NAEP  consultants  have 
expressed  the   feeling   that   these   skills   should  be  measured 
by  using  actual  writing  samples.** 

Time  Constraints >     As  with  the  t e acher- c r it ic s ,  the 
question  of   timed  tests  was   raised.      Ed  White  mentioned  the 
problem  of  ''adequate  time"   for   certain  kinds  of  writing  on 
timed   tests,    and  said,   "We  still  need  discussion  of  appro- 
priate  time  allotments   for  different   tasks  and  different 
students."     Brossell   said  that   generally  there   is  too  little 
time  provided,   especially  on  college-level  tests.  But 
Brossell  also  went   on  to  state   that   there  needs  to  be  more 
made  of 

the  great   differ en ce  between  the  writ  in g   situation  in  a 
classroom  and   that   in  a   test.      Good  writing  programs 
(and  good  writing  classes)   allow  writers,   even  encourage 
them,   to   discover  what   they  want   to  say  and  give  them 
time  and   guidance   in   saying  it.      I  submit   that   this  is 
the  antithesis  of   formal  testing,  where  a  writer  is 
given  one  chance  to  do  his  best  under  constraints  of 
both  topic  and   time.      Also,    there's   a  difference  between 
large-scale  testing  and  evaluating  writing  performance 
in  a  classroom--especially  from  the  writer's  point  of 
view. ...The  psychological  effects  of  the   testing  situa- 
tion in  my  view  are   dramatically  different   from  those 
of   other  writing  occasions.      The  test   is   an  isolated 
event   in   the  lives  of  writers,   often   fraught  with 
diabolical  significance.      Its   immediacy  and  moment  are 
quite  imposing,   even  distracting.     Just   as  a  football 
player  practices  his  plays  diligently,   then  fumbles  four 
times   in  the  big  game,   so  a  writer  can  know  all  his 
moves  but   fail  to  execute  under  the  pressure  of  a  test. 
Tests  need  to  be  constructed  with  this  kind  of  distinc- 
tion in  mind--the  distinction  between  "practicing"  and 
"playing" ....  Teaching  writing  and  testing  writing  are 
much  more  distinct   than  they  appear. 

Use  of  Pilot   Tests.      Several   of  these  respondents  also 
mentioned  the   importance  of  pilot   testing.      Brossell  stated 
that  pilot   testing   is  essential  in  the  development   of  prompts — 
it   provides   empirical  evidence   for   the   effects  of  the  prompts 
on  writers.      But  he  warned  that  pilot   testing  doesn't  assure 
one  that  a  prompt   is   doing  what   one  wishes   it   to  do. 
Interestingly  enough,   one  of  the  professional  evaluators, 
Ann  Humes,   not   only  agreed  with  the  need  to  pilot   test  all 
prompts,   but    she  also  suggest ed  pilot   testing  with  teacher- 
colleagues   first,    and  then  with  stud en ts. 

Effects  of  Recommendat  ions   on   the   Final  Version 

Recommendations  by  Teacher  Reviewer s ,     Most  of  the 
teacher-reviewers  Inade  specific^  recommendations  for  changes 
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•-in  the   final  version  of  the  Keech  manuscript   on  "Practices  in 
Designing  Writing  Test  Prompts."     Many  of  the  suggestions 
were  incorporated  into  the   final  text.      Some  examples  follow: 

Mary  Frances   Claggett,    a  high   school   teacher,  suggested 
that   there  be  more  examples   of  under-   and  over-prompting  and 
other    forms   of   unhelpful   constraints   which  practitioners 
could  use  as   a  reference   in  their  own   development   of  prompts. 
Such  examples  were  included   in  the   final  version. 

Bill  Robinson  noted  that  writing  about  an  imaginary 
situation  "can  produce  the  Great  Conditional  Problem,"  and 
suggested  that  this  idea  be  expanded.  The  final  version 
discusses  the  problem  of  the  conditional  tense  with  regard 
to  imaginary  situations,  and  proposes  specific  wordings  of 
this  kind  of  prompt  in  order  to  avoid  eliciting  the  condi- 
t  ional . 

Tanner   suggested  providing  more   examples   of   responses  to 
sample   prompts,   and  this,   too,  was  done  to  a   limited  extent 
in  the    final   version   of   the   report.      However,    samples  of 
responses   to   prompts   are   to  be   found   in   later  chapters  of 
the  full  report.      See  particularly,    the  chapters   relating  to 
the   studies   done  at   Los   Gatos   High  School  and   for   Santa  Clara 
Coun  t  y . 

Tanner   also   suggested  providing  an   ill-conceived  prompt 
and  then   showing  its   revision,   with  reasons   for  the  revision 
and  explanations   for   one  working  and   one  not   working.  Along 
these   same   lines,   Bernstein   suggested   that    including  sample 
rubrics  would  be  useful.      Although  she  agreed   that    it  is 
important   to   discuss   papers  at    length  before   scoring  them, 
she  said  it   was  also   important   to  have  written   criteria  for 
determining  scores.      In   this  context,    she  made  an  interesting 
observation   about   different   rubrics   being  needed   for  compe- 
tency tests,    proficiency  tests,   and  advanced  placement  tests, 
because  of  the   different  natures  of  each  kind  of  test;  that 
is,   each  of   these  represents  a  different  kind  of  testing 
intent.      The  Tanner   suggestion  was  acted  upon.      Poorly  con- 
ceived prompts  are   introduced  with  recommended  revisions  and 
discussions   of  the  problems.     Ruby  Bernstein's  recommendation 
seemed   to  go  beyond   the  scope  of   this  phase  of  the  study,  and 
thus    it   was   not   acted  on. 

An   interesting   suggestion  came  from  four  of   the  eight 
teacher-critics  who  responded.     Farrell  suggested  ^n 
heuristic   in   the  form  of  a  set   of  questions   that  any  teacher 
or  test  maker  might   ask  himse 1 f /her se 1 f  about  a  writing 
prompt."     Such  questions,   Farrell  noted,   could  also  be  an 
efficient  way   of   summarizing  the  major  points  made   in  the 
study.      Tanner  also   saw  the  need   for   an  heuristic: 

What  are  the  questions  that  a  writer  of  a  prompt  should 
ask  himself  as  he  tries  to  write  a  prompt  for  a  certain 
type  of   test?     What   are  the   issues?     How  do  these  issues 
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relate  to  the  needs  of  the  reader  who  will  be  trying  to 
read  the  written  responses  and  sort  out  the  levels  of 
success  these  who  show  toward  the  target  objectives^... 
I  would  try  to  list   these  questions  and  concerns.  I 
would  try  to  relate  these  concerns  to  the  language  of 
the  prompt,   and  to  the  purposes  of  the  test. 

On  the  practical  level.  Bernstein  suggested  an  anecdotal 
example  of  a  school's  or  a  department's  step-by-step  procedure 
for  developing  a  prompt— how  the  teachers  went   about   the  for- 
mulation of   it,    from  start   to  finish,    including  pitfalls  and 
problems.      Claggett   underscored  the  need  for  such  an  example 
with  her  own  recounting  of  how  her  district  writing  center 
has  used  "brainstorming"   to  come  up  with  prompts.  Their 
trial-and-error  approach  to  using  them  has  had  only  a 
success  rate.      Some  sort   of  heuristic  would  be  better  than 
this   "haphazard  approach,"   she  said. 

At   least  three  sets  of  guidelines  have  been  included  in 
this   final  report.      In  an  appendix  to  Part   II  there   is  the 
set   of  Guidelines   for  writing  prompts   devised  by  Gertrude 
Conlan,   Chief  Reader   for  the  Educational  Testing  Service. 
In  Part   I,    there   is  a  synthesis  of  principles  for  construct- 
ing prompts  derived  from  the  recommendations  of   teachers  and 
scholars.     There  is  also  in  Part  I  a  set  of  guidelines  f|j 
writing  prompts  based  on  a  recorded  session  where  three  ^ 
teachers  met   to  develop  an  assessment  prompt   for  an  actual 
assessment • 

Robinson  suggested  that   because  many  teachers  «nd  test 
administrators  might  not   be   familiar  with  "testing  talk, 
there  should  be  included   in  the  study  some  brief  definitions 
of  certain  terms,   especially  "valid"  and  "reliable.  with 
emphasis,   as  he  put   it.   on  "the  inh^ent  """^^-^f J^^^ 
essay  tests"  to  measure  writing  ability.     Along  these  same 
lines,  he  said  that  a  section  was  needed  on     ways  in  which 
the  reliability  of  the  essay  test  can  be  increased--or  the 
unreliability  compensated  for." 

Keech  has   included  in  her   final  version  of  Part   II  a 
definition  of  key  terms,   and  throughout  Parts   I  and   II  there 
are   discussions  of  the  limitations  of  various   forms  of  meas- 
urement  of  writing  competence. 

A  final  suggestion   for   improving;  the  .design  of  writing 
exercises  came   from  Jean  Jensen,  who  suggested  adoption  of 
the  rule   followed  by  the  teachers  in  her  school:     If  the 
papers  you  get   .re  a  disappointment,   then  what   you  wanted 
was  not  what  you  asked  for.     Discuss  the   'topic'   with  the 
class?"     One  of  the  main   studies   in   the  Writing  Assessment 
Project  was  devoted  to  investigating  the  match  between^the 
task  expectations  of  the  tester  and  the  student  writer  s 
interpretation  of  the  task  provided.     The  results  of  this 
investigation  are  presented   in  Technical  Report  No.  4. 


ERIC  13;. 


Recommendations  by  Researchers   and  Professional 
Evaluators ,      Fewer   specific   additions  were  suggested  by  the 
researchers   and  professional  evaluators;   for  the  most  part, 
they  responded  to  what  was  there.      Charles  Cooper  suggested 
that   more  illustrations  be  provided  of   specific   points,  and 
this  was  done   in   the   final  version.      Rex  Brown  suggested  that 
'•it   might  be  useful  to  explain  some  of   the  problems  and  traps 
in  terms  of   the  end  of   the  proces s--holist ic   scoring — rather 
than   the  beginning.     Perhaps  you  could  present   a  description 
of  the   ideal  pool   of  papers   for  an   ideal  holistic   scoring  by 
ideal   scorers,   and  then — having  established   the  principles 
of   the   end  process — work  backwards   toward  the   principles  of 
topic   creation."     Somewhat   along  these  same   lines --and 
similar  to   the  suggestion  made  by   several  of   the  teacher- 
critics--Ed  White   stated  that   "the  most   important    lack... is 
a  description  of   the  process  of  question  development."  Ann 
Humes   likewise  recommended:   "Use  a  heuristic   device   in  writing 
a  prompt   to  make   sure  you  have   covered   the   relevant  rhetorical 
elements;   e.g.,   A.   what   is   the  purpose    (why?),   B.   what   is  the 
subject /topic    (what?),   C.   who  is   the   audience    (who?),   D.  what 
is   the   form/genre  (how?)." 
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MORE  RESULTS   OF  THE  STUDY 


The   following   set   of   guides  for  designing  writ- 
ing test   prompts  are  presented  here  with  the  possi- 
bility that  they  may  aid  their  users  In  developing 
understanding  of  the  nature  and  range  of  effect  that 
a  prompt  may  evoke.     We  need  to  do  our  best   to  create 
prompts  that  at   least  touch  the  edge  af  the  student 
writer's  world  knowledge.     We  need  to  offer  the© 
starting  points  for  composing  that  enable  the©  to 
move  freely  within  whatever  kinds  of. guiding  frame- 
works are  given.     We  must   be  sure  that   the  prompt  is 
couched  to  forestall  the  needless  difficulties  that 
arise  from  ambiguous  language  and  confusing  signals. 
We  may  never  write  the  perfect   prompt,   but  pupils 
deserve  to  meet  year-by-year  assessments  prompts  that 
are  as  exactingly  principled  as  possible  In  their 
design . 


Guides  for  Designing  Writing  Test  Prompts 

The  following  guides  are  presented   in  the  state 
of  development   they  had  reached  by  the  end  of  the 
Writing  Assessment  Project.     They  await   further  collab- 
orative refinement  as   fellow  professionals   test   them  in 
practice.     The   set   below  includes: 

1.  The  Philosophy  of   the  Assignment   for  Writing  Assessment 

2.  Guidelines   for  Developing  Topics   for  Writing  Assessment 

3.  A  Tentative   Schema  of  Writing  Tasks   for   the  Development 

of  Assessment  Prompts 

4.  Schema   for   Task  Analysis  of  Prompts 
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TH£  FHILOSOPHY  OF  THE  ilSSICNMENT  FOR  WRITING  ASSESSMENT* 


Maxim:    A  v«ll*-frAacd  assignment  for  •  vriting  assessment 
provides  an  occasion  for  writing  in  vhich  the 
student*a  powers  of  expression  and  connunication  are 
stimulated  to  their  nbrval  maxlmun. 

Frinciples: 


1.  The  subject  chosen  must  be  potentially  interesting 
to  the  tescher-reader  or  evalustor  of  the  esssys 
written. 

2.  The  subject  chosen  must  be  potentially  Interesting 
to  the  student  writers. 

3.  The  assignment  furnishes  data  cu  atart  from  which 
is  open  to  presentation  In  any  of  several  forms 
ranging  from  statements  or  questions  to  pictures 
or  cartoons  accompanied  by  Instructions. 

4.  The  assignment  must  be  meaningful  within  the 
student's  experience. 

5.  The  assignment  seeks  to  elicit  a  specific 
response  and  limits  content  or  form  or  both. 

6.  The  assignment  is  itself  specific  In  suggesting 
an  audience  beyond  the  teacher  or  evalustor  when 
the  subject  aaV.ss  such  specif icstion  feasible. 

7.  Assignment  by  topic  slons  Is  srtificial  snd  yields 
s  Isck  of  focus  in  development. 


Coda:    A  vell-frsmed  sssignasnt.  clssr  and  prsdse  In  Its 

Instructions,  shows  svsreness  of  the  sudience  of  wsiting 
examinees.    The  demands  on  the  assignment  creator  sre 
identical  with  those  on  the  writer. 


*A  Bynthmgig  of  prof^MMion^X  wimdom  hassd  on  my  import 
on  SOUnCES  OF  KNOWLEDGE  FI^OM  TMEOKY  AND  RESEARCH  FOR 
DESIGNING  WRITING   TEST  PROMPTS.      NIE/BAWP  Writing 
AsssssmenC  Projmct,  VnivzBity  of  CAlifoznis,  *srAei#y 
I9$2 
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CUIDEl.mS  FOR  DEVELOPIKC  TOPICS  FOR  WRITING  ASSESSMENTS^ 


(These  guidelines  are  based  on  a  copic  developmenc  session  wich  Ruby  Bemscein, 
Clairt  Pelcon,  and  Bill  Thomas  parcicipacing.    BAW?  Uricing  Assessmenc  Project 
Study  of  Topic  Development.    University  of  California,  Berkeley,  May  1981.) 

Sone  Questions  to  Consider  in  Creating  Topics  for  Writing  Assessments 

1,  Why  arc  we  doing  this?    What  Is  the  purpose  of  this  writing  assessment? 


2.  What  will  we  do  with  the  results? 


3.  What  is  the  nature  of  the  student  population  to  be  assessed? 

«•  What  is  their  grade  level? 

b.  What  is  their  general  level  of  academic  competence? 

c.  What  prior  curricular  experience  have  they  had  that  prepares  them 
for  this  assessocnt? 

4.  What  is  the  aost  appropriate  Copic  subject  for  this  assessment? 

a.  What  prior  knowledge  will  discussion  of  this  subject  require? 

b.  Is  this  subject  accessible  to  a  wide  range  of  students? 

c.  What  Is  the  relation  of  the  topic  subject  to  the  students*  world? 

Is  it  connected  to  real  student  concerns? 
Is  it  remote  or  alien  to  their  world? 
Is  it  an  Interesting  or  involving  topic? 

d.  Are  there  aspects  of  the  subject  that  may  lead  to  stereotyped  thinking, 
reliance  on  cliches? 

e.  Is  there  any  element  of  the  subject  that  may  be  considered  unduly 
threatening  to  personal  values,  religious  beliefs,  self-esteem,  etc.? 

f.  Is  the  subject  free  fron  biasing  effects  for  different  sub-groups 
within  the  student  population? 

Is  it  too  limiting  for  upper  level  ability  students? 

Does  it  presume  experience  that  «ay  have  been  denied  some  students 

for  social,  cultural,  or  economic  reasons? 
How  difficult  Aight  the  task  be  for  the  lowest  twenty-five  percent 

of  the  student  population? 

g.  Will  the  topic  lead  to  diatribes  against  persons  at  che  school,  parents, 
local  citizens? 

h.  How  is  the  subject  related  to  the  school  curriculum  and  can  equal 
exposure  to  similar  curriculum  experiences  be  assumed? 

i.  How  does  the  choice  of  subject  affect  choice  of  rhetorical  aode? 

5*  How  should  the  topic  subject  be  worded? 

a*  What  do  the  verbs  used  communicate  to  the  student?    e.g..  Describe? 

Tell?    Explain?  Illustrate?    Analyze?  Discuss? 
b«  Wh^t  do  the  commonly  used  formal  labels  actually  connunicate?  Essay? 

Story?  Argument?    Persuasive  essay?  Letter?  Journal  entry? 

c.  Should  a  range  of  fonuil  options  be  proposed? 

d.  To  what  extent  should  students  be  cued?    e.g.  **Write  about  one  thing...** 
"Tell  about  three  improvements..." 

e.  What  level  of  background  detail  should  be  provided? 

f.  Should  the  subject  include  specification  of  an  audience? 

6.  How  should  instructions  be  handled?    Embedded  in  the  topic  or  kept  separate? 
Should  the  topic  contain  hints  about  approaches  to  the  subject  or  suggestions 
for  task  management? 


^Prepared  by  Leo  Ruth 
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A  TfNTATIVE  SCHEHA  OF  WRITING  TASKS  FOR  THE  DtVELOWCNT  OF  ASSESSMENT  PiOMPTS^ 


GENERAL  URiTING  PURPOSES 

1.  To  describe 

2.  To  narrate 

3.  To  record/report 

4.  To  explain/direct 

5.  To  persuade 

To  request 

7.  To  plan 
S.  To  edit 


WRITTEN  PRODUCTS  TO  BE  ELICITED 

1.1  Sustained  description  of  person,  place,  or  object 

1.2  Description  and  expression  of  feelings  toward  what  Isflescrlbed 

1.3  Sequenced  description  fro«  mcnory 

1.4  Description  to  accompany  9<»en  pictures 

2.1  iMQlnatlve  narrative  based  on  given  characters  and  stting 

2.2  Original  end  to  a  story  selected  by  the  pupil 

2.3  Short  story 

3.1  Autobiographical  account  of  event  experienced  recently 

3.2  Verifiable  account  of  an  event 

3.3  An  account  of  soaething  learned 

4.1  Explanation  and  reflection      on  a  regulation 

4.2  Explanation  of  a  coaplex  skill 

4.3  Instructions  to  acconpHsh  a  simple  household  task 

5.1  InforMl  letter  to  a  friend  to  get  reader    to  change  Bind 

5.2  Argument  Justifying  a  point  of  view 

6,1  Letter  to  a  person  In  a  public/private  agency 
6 J  Letter  of  application 
7.1  Account  of  an  activity  to  be  undertaken 
6.1  Editing  of  a  written  account 


Note:    All  tasks  envision  provisions  for  variation  In  audience  and  the  writer's  control  over  appropriate  for«s. 
The  framework  provides  for  attention  to  four  dimensions: 

1.  Variations  In  mode:    narrative/descriptive;  reflective/analytical 

2    Degree  of  control  granted  writer  or  assessor:    ranging  from  open  to  closed  specifications 

3.  Source  of  subject  matter:  first-hind  experience  or  knowledge  from  elsewhere 

4.  Range  of  purposes:  functional  and  literary  .  ^    i  ^  * 
It  Is  expected  that  analytic  marking  following  upon  an  Initial  holistic  scoring  would  take  Into  account 
the  variations  In  content,  structure,  style,  and  grammar  that  different  writing  tasks  Impose.    It  Is 
also  assumed  that  topics  would  be  conceived  under  the  general  philosophy  accompanying  this  schema. 

*  Adapted  from  Gorman,  T.P.,  etad.    Language  Performance  in  Schools:  Prlmarij 
Survey  Report  No.  1  (1981),  p.  85,  and  Secondary  Survey  Report  No.  1  (1982), 
pp.  66-67.    London:    Her  Majesty's  Stationery  Office.    These  two  reports  pre- 
sent the  results  of  the  1979  assessment  conducted  by  the  Language  Monitoring 
T^.am  at  the  National  Foundation  for  Educational  Research  in  England  and  Wales.  ' 
Leo  Ruth,  NIE/BAWP  Writing  Assessment  Project,  University  of  California, 
Berkeley,  1982. 


SCHEMA  FOR  TASK  ANALYSIS  OF  PROHPTS* 


LEO  RUTH 


PRQULEN:  HIiat  range  of  iNFOflmrioN  is  required  in  a  writing' task  statdent  to  enable  a  mriter  to  proceed 

WITH  PREClSiat-TO  RECOGNIZE  AM)  ACC(m.!SH  TIC  TASK  SPECIFIED  IN  A  MANNER  EXPEaEO?  TasK 
SPECIFICATION  IS  VIEICO  AS  A  PROCESS  OF  THE  REDUCTION  OF  UNCERTAINTY. 

CAIEGQRlf  DPRIGR  Mm£DGE  2)GEII€  imiQii 

PARADIGM  Drahing  on  m  ntppwimg  m  iMAGiNATifw.  write  SfitEDiUfi  wour  sarar  n»  scMnut  fe 

1.  

2.   [  

CAWm      t)«l)IBCE  5)RDl£  6)SWrAX  7)fUlCTIQN 

PARADIGM  TO  snatj    OTUMim  any  voicf  or  rfgistfb  in  tttt  SHLL  for  m  pibposf  or  usf 

1.  _   

 —  p  .  ■  ,  ~-  ' 

2.   '   


OflEQDRir  8)aiflMrnY  9)ICSULT 

PARADIGM  AT  ANY  Iftfim  TO  ACHIEVE  tW  EEEECL 

1.  

2.   ;  

TOPIC/TASK  STATBfffT:  


*NIE/B»y  Areji  Writing  Project  Writing  Assessment  BroJ^ct,  University  of  C       fornid , 

f  A  Berkeley 
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Chapter   1:     Part  II 


PRACTICES   IN  DESIGNING  WRITING  TEST  PROMPTS: 
ANALYSIS  AND  RECOMMENDATIONS* 

Catharine  Keech 


IHTRODUCTIOH 

Tht  rtctnt  tffort  to  Introduet  dirtct  atafurts  of 
wrltmi  ptrforaance  Into  loci  *#rltlnt  assessatnts  has 
found  local  ttachtrs  and  adalnlatrators  -^th  f«- 
raaourees  to  guide  thta  In  dtvtloplnf  good  aaaay  testa. 
While  there  la  an  eitenslve  nttrature  dtscrlblnf  pro- 
oedure.  for  creating  valid  and  reliable  objective  teat, 
of  certain  kinds  of  learner  achleveaent.  there  la  very 
little  m  the  May  of  aclence  to  guide  the  ^tvelopaent 
of  p.Jfor.anoe  teata.  eapeclally  teata  of  -rlt  ng  abil- 
ity.    This  la  not  to  say  »e  do  not  have  a  »''0''Jn*  .  ^ 
literature  on  hoM  to  acore  -rltlng  aa.plea.  -ith  advo- 
cates of  several  coftpetlng  aethods  addreaalng  JJe.- 
selves  to  fo.e  ettent  to  the  proble.  of  developing  good 
prompts  or  writing  stl.ull  with  Instruction,  for  the 
S^JJ.r.     in  fact,  perhaps  the  .ost  eitenslve  -ork  tak- 
ing into  account  the  relation  of  the  teat-topic  to  the 
scoring  plan  h.s  been  done  by  dy«Jop«r«  of  -J**  Is 
called  Prl-ary  Trait  Scoring.  ( Hullls . 197J;  ^loyd- 
Janss     1977).     But  ss  *#lth  Other  approaches,  the  tltl- 
if  tJis  rpjr;ach  to  ass.ss-ent  Identifies  It  as  ..Inly 
a  scoring  sppro.ch.     ETS  test  developers,  l^/* 
eviiluators  atte-ptlng  to  develop  good  direct 
of  writing  ability,  are  willing  to  ad.lt  'or  the  .oat 
part  that  writing  goad  toplca  re.alna  •or«  •«  •'"t  than 
a  science.     Up  to  now.  deciding  when  a  *oplo  la  good 
has  been  typically  .ore  Intuitive  than  objective. 

Thla  report  ralaea  a  queatlon  baalc  to  the  conduct 
of  writing  assessment:  How  should  students  ba  "J^*^  *o 
;;ite  and*what  ahould  they  be  aak.d  to       Jj. •Jj"*^! 
we  wlah  to  .easure  their  real  writing  abllltiea?  The 
aaatined  writing  taak  playa  a  critical  role  In  deter- 
mining how  atudenta  perfor.  and  whether  teatara  are 
■eaaurlng  what  they  think  they  are  .easurlng. 

Creating  a  good  pro.pt  Is  not  easy. 
write  best  when  they  write  with  Intention.    That  la. 
they  .uat  be  able  to  find  ao.ethlng  they  want  to  aay  to 
soseone.     However,  -intention-  la  "ot  the  .a.,  aa 
-.otlvatlon.-  Mo  .atter  how  highly  .otlvat.d  they  .ay 

TTi7r;-repori  ia  baaed  upon  reaeareh  supported  by  the 
y.*4««.i  Inatitute  of  Education  under  Grant  Mo.  MIE  C- 
SO-OOS;  to  tS"B:5  A?;a  Writing  foject.  Unlveralty  of 
California.  Berkelay.  Leo  »uth.  Project  Director. 
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be  to  perfora  wdll  on  a  writing  test,  no  aatter  how 
conserned  they  are  to  achieve  a  good  score,  they  will 
not  be  able  to  do  their  best   if  they  find  the  topic 
dull,  confusing,  or  in tiaid a  ting •     They  need  to  seize 
on  the  gera  of  an  idea  and  begin  writing  with  confi- 
dance  if  they  are  to  t*"*'**^^  *  cotaplete  piece  of  writ-* 
ing   in  a  Halted  aiaount  of  tine.     Tne  Intent  to  commun- 
icate releases  abilities  which  motivation  leaves 
untapped.* 

In  creating  a  writing  prompt,  the  tester's  main 
purpose  is  not  to  single  out  clever  students  who  can 
decipher  complicated  instructions,  but  rather  to  make 
it  easy  for  as  many  students  as  possible  to  respond 
enthusiastically t  demonstrating  their  best  writing 
abilities.     The  test-maker  cannot  always  be  sure  that 
students  will  interpret  the  task  as  intended  and 
respond  appropriately:  sometimes  a  single  word  will 
mislead  a  wr  1  ter-<»aometimes  instructions  are  inadequate 
to  inspire  a  sustained  piece  of  writing.     Testers  need 
to  be  sure  that  every  student  will  be  able  to  respond, 
but  they  are  also  interested  in  whether  the  kind  of 
writing  elicited  will  allow  measurement  of  the  abilU 
ties  they  wish  to  evaluate.     Different  kinds  of  prompts 
elicit  very  different  kinds  of  writing — some  students 
do  better  at  one  kind  than  another;  some  teachers  and 
administrators  are  more  interested   in  one  kind  than 
another. 

In  addition,  while  a  classroom  teacher  can  discuss 
the  assignment  with  students,  check  their  understanding 
of  the  taskt  and  provide  additional  instructions  as 
needed,  the  test-maker  must  write  the  assignment  in  a 
form  that  is  as  nearly  fool-proof  as  possible.  The 
assignment  in  a  test  often  must  guide  the  writing  of 
students  from  many  different  classrooms,  including  stu- 
dents accustomed  to  different  interpretations  of  such 
commands  as  "deacribe,"  or  **write  a  short  essay 
about....**     The  teat -maker  has  recourse  to  several 
strategies  to  offaet  these  difficulties,  the  most 
important  of  which  is  pilot-testing  the  writing  assign* 
ment  or  checking  on  its  workability  with  a  sample 
group  .     (Kirr  ie  ,    1980 . ) 

The  following  review  of  practices  in  designing 
prompts  aims  to  identify  baaic  principles  that  guide 
professional  test -makers,  to  alert  novice  test-makers 
to  potential  problems  in  prompt  design,  and  to  eiplore 
the  relationship  between  the  nature  of  the  writing  t^st 

•S«e  Woodward  and  Keech,  1930  (pp.  6-10)  for  an  extend- 
ed discussion  of  the  distinction  between  motivation  and 
in  ten  tion . 
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and  the  purposes  of  the  assessaent.     The  first  section 
clarifies  terms  used  in  the  discussion  of  writing 
assignments  and  distinguishes  between  two  basic  parts 
of  a  prompt:   1)  the  topic  or  other  writing  stim^ulus 
and,   2)  the  instructions  to  the  writer  > 

The  second  section  focuses  on  -the  nature  of  'he 
topic  or  other  stimulus.     It  relates  the  choice  of 
topic  to  the  test  purposes  and  the  kinds  of  writing 
desired,  as  well  as  pointing  out  several  common  prob- 
lems  for  topic  writers. 

The  third  section  deals  with  the  Instructions  to 
the  writer  and  how  these  relate  to  the  criteria  by 
which  the  writing  Is  to  be  Judged,  as  well  as  to  the 
test-maker's  decision  about  what  Is  and  Is  not  being 
tested.     It  relates  the  wording  of  these  Instructions 
to  the  purposes  of  the  assessment,  and  It  points  out 
some  common  problems  In  giving  Instructions  to  writers. 
k  list  of  suggestions  for  writing  essay  prompts  and  for 
Judging  the  worth  of  a  writing  prompt  as  prepared  by 
Gertrude  Conlan  of  the  Educational  Testing  Service  Is 
Included  In  Appendix  A.     A  list  of  the  writing  tasks 
discussed  In  this  report  with  their  sources  appears  as 
Appendix  B.     References  for  this  report  follow  the 
appendices . 
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THE  NATURE  OF  THE  rfRITIMG  PROMPT 


L  NQ^^  on  Teras 

Multiple  choice  tests  of    verbal  skills  are  often 
labeled  writing  tests  or  even  coaposl tlon  tests 
although  no  writing  or  ooaposlng  is  involved.     In  this 
report,  however,  the  tera,  writing  test,  will  refer 
only  to  those  tests  which  call  for  one  or  aore  saaples 
of  student  writing.  This  general  tera,  writing  test,  is 
preferred   to  the  traditional  use  of  the  eipression 
essay  test ,  which  is  reserved  here  for  those  writing 
tests  which  call  on  students  to  produce  essays,  per  se. 
The  tera,  wri ti  ng  saaple ,  refers  to  the  writing  pro- 
duced for  writing  tests,  whether  by  a  single  student  or 
by  an  entire  student  populationt     (In  this  report, 
there  is  an  alternation  of  reference  to  studen t^wr i ter s 
and  writers  though  the  full  tera  is  always  intended.) 
Hence,  the  designing  and  adainister ing  of  the  writing 
test  is  followed  by  the  collecting  and  scoring  of  the 
writing  saaple .     (Generally,  those  who  score  the  wri t^ 
ing  saaples  are  siaply  called  readers ,  but  in  order  to 
avoid  confusions  with  other  coaaon  usages  of  this  terat 
the  aore  cuabersoae  fora  teacher^-readers  is  used  fre- 
quently in  this  report.)  These  personnel  and  procedures 
together  constitute  the  holistic  writing  assessaent . 

The  invitation  to  write  in  a  writing  test  has  been 
referred  to  variously  as  the  topic,  the  task,  the 
proapt ,  the  test  question ,  the  writing  stiaulus ,  the 
writing  instructions,   the  assignaent .     In  this  report, 
soae  or  these  teras  will  be  used  interchangeably  — 
especially  the  writing  task,  the  proapt,  the  test 
question-^  to  refer  to  the  whole  writing  assignaent. 
That  assignaent  generally  consists  of  two  aain  parts: 
1)   the  topic,  subject,  or  stiaulus-«the  tester's  iden- 
tifioation  of  what  the  writer  will  write  about;  and  2) 
the  instr uc tions--the  particular  suggestions  about  or 
liaitations  on  the  content  of  the  writing,  and  the 
tester's  constraints  on  how  the  writer  is  to  address 
the  subject,  with  any  specifications  on  what  the  writer 
is  eipected  t^  do. 

The  Structure  of  the  Proapt :     Topic  and  Instructions 

In  designing  the  writing  proapt,  the  test-aaker 
aust  choose  or  Invent  topics  or  other  stiauli  that  pro- 
vide students  with  soaethln|  to  write  about ;  foraulate 
instructions  for  writing  which  help  students  perfora 
the  writing  task  to  the  best  of  their  ability;  and 
ensure  that  the  topic  and  the  instructions  work 
together  to  elicit  the  particular  writing  skills  the 
tester  is  interested  in  aeasuring.     He  shall  consider 
first  the  selection  of  topics  and  then  the  role  of 
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instructions  with  oth^r  types  of  stiauli* 

In  a  writing  test,   the  student's  eoiaposition  is 
expected   to  address  the  given  topic,  and   it  will  be 
judged   in   part  on  how  well  it  addresses  that  topic* 
The  student's  response  to  the  topic  will  be  natched 
against  the  interpretation  of  the  topic  expected  by  the 
evaluator.     All   topics  can  be  considered   proapta  for 
writing,   but  not  all  writing  proapts  provide  a  fully 
foraed  topic  for  the  writer  to  develop  in  to  an  essay. 

When  the  proapt  is  just  a  word  or  a  phrase,  it 
requires  the  student  to  generate  a  predication  which 
foras  the  topic   into  a  subject  for  developaent. 

Josephine  Miles  (1977)  has  noted  that  the  giving 
of  a  topic,  in   the  sense  of  the  giving  of  a  subject  to 
write  about for  exaaple,  "Hy  Hoae  Town"-«^  does  not 
provide  the  writer  with  a  theais  or  a  generalization 
which  aight  generate  a  coherent  essay.     The  writer  aust 
discover   his  own  thesis  by  shaping  a  predication  for 
his  subject  that  is  susceptible  to  deaonstration  and 
developaent;  for  exaaple,  ''My  hoae  town  provided  a  rich 
environaent  for   the  developaent  of  a  bad  character." 

The  conception  of  "topic''  which  requires  students 
to  generate  their  own  predications  is  exeaplified   in  A 
Thousand  Topics  for  Composi  t  ion ;     Revised  ,  collected  in 
the  Illinois  English  Bui leti  n"Tl 97 1  )  t  or   in  the  aore 
recently  published  What  Can  I  Write  About?  Seven 
Thousand  Topics  for  High^School  Students  (Powell, 
1981).     Most  of  these  topics  are^ahort  phrases  which 
aight  serve  as  titles  of  student  essays:  "ay  hoae 
town,"  "a  childhood  aeaory,"   "ay  favorite  object,"  "sex 
education,"  "capital  punishaent,"  "aaklng  tacos,"  "the 
case  for/against  dieting." 

The  nature  of  soae  writing  tests,  however,  aakes 
it  aore  appropriate  to  apeak  of  a  "stiaulus"  rather 
than  a  topic  for  writing.     The  stiaulus  for  writing 
could  be  a  faaous  quotation,  a  coaplete  poea,  a  contr- 
oversial stateaent,  a  fragaent  of  an  essay  or  story,  a 
photograph  or  drawing,  a  piece  of  ausic,  a  taped 
conversation,  a  fila-clip,  a  news  bulletin.     In  these 
cases,  the  instructions  to  the  writer  rather  than  the 
stiaulus  alone,  will  deteraine  how  the  proapt  aerves  to 
elioit  writing.     The  atudent  writer  aight  be  directed 
to  respond  to  a  photograph,  for  instance,  by  creating 
an  iaaginative  piece  which  recounts  a  personal  experi- 
ence, relates  a  streaa  of  consciousness,  or  tells  an 
invented  atory  with  no  reference  whatever   to  the  photo- 
graph Itself.     But  when  the  atudent  is  asked  to 
describe,  coaaent  on,  criticize,  evaluate,  agree  or 
diaagree  with,  explain,  or  in  soae  way  to  write  about 
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the  given  stloulus,  then  a 
Indicated  and  the  stloulus 
full  proapt  for  writing. 


line  of  development  has  been 
plus  instructions  become  the 


For  younger  children  or  for  tests  of  general 
fluency,  very  slaple  phraslngs  of  both  topic  and 
Instructions  aay  be  quite  effective  in  eliciting  Inten- 
tional^ and  coherent  writing.  For  ezaaple: 

PROMPT  la 

Tnink  about  the  word  trees .     Write  about  what 
the  word  suggests  to  you.* 


Note  that  the  topic  points  to  one  word:  "trees."  In 
this  case*  the  Instructions  to  the  writer  are  quite 
simple  and  provide  almost  no  constraints:  **Think  about 
the  word...**;   **tfrit<i  about  wtiat  the  word  suggests....** 
Tne  same  topic  could  be  accompanied  by  more  elaborate 
instructions  which  would  encourage  students  to  move  out 
of  writing  only  for   self-ex presslon  toward  writing  for 
literary,  or  even  for  academic  or  social  purposes: 

PROMPT  lb 

Think  about  the  word  trees .     Write  a  story  in 
which  trees  play  an  important  part.     This  may 
be  a  real  experience  or  an  imaginary  story. 
Tou  may  even  write  as  If  you  were  a  tree.  Be 
sure  that  trees  play  some  part  in  your  story. 


PROMPT  1c 

Think  about  the  word  trees .     Write  an  essay 
in  which  you  explain  one  of  several  possible 
ideas  this  word  may  suggest  to  you.  You  may 
write  about  a  special  kind  of  tree,  a  special 
use  of  trees,  people's  attitudes  toward 
trees,  or  something  else  that  interests  you 
about  trees.     Focus  on  one  idea — do  not 
attempt  to  list  all  the  things  trees  can 
mean  . 


In  the  above  two  examples,  it  is  not  the  topic  but 
the  writing  instructions  that  determine  whether  the 
test  Is  one  of  general  fluency  calling  for  self- 
expression  and  free  exploration  of  internal  feelings. 


*A11  credits  for  prompts  appear  in  the  Prompt  Index, 
Appendix  B. 
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or  whethtp  It  Is  a  test  of  nor*  rtflned  coaposlng  stra- 
tegies m  which  specific  foras  or  discourse  conventions 
(#111  be  rewarded.     In  exaaple  la,  Instructions  are 
brief  and  very  staple;  In  exaaples   lb  and   1c,  Instruc- 
tions have  been  more  fully  specified. 

With  a  single-word  proapt,  the  Instructions  chosen 
by  a  tester  can  be  repeated  froa  year  to  year,  test  to 
test,  without  changing  the  deaands  of  the  task,  slaply 
by  replacing  the  stlaulus  word,  with  equally  faalllar 
terat  for  the  population  tested,  for  example,  aaaks, 
scars,  plastic ,  sports,  work,  play,  fire.     The  acre 
open  to  Individual  Interpretation,  the  acre  likely  the 
topic  Is  to  elicit  Intentional  writing  froa  students. 
The  wide  range  of  responses  elicited  by  such  open 
topics  also  aakes  interesting  reading,  eoabatlng  the 
reader  fatigue  that  soaetlaes  results  when  hundreds  of 
predictably  slallar  essays  aust  be  scored.     The  next 
section  discusses  the  selection  and  wording  of  the 
writing  topic  to  apeclfy  the  content  of  the  essay. 
(Methods  of  selecting  and  presenting  quotations, 
literary  passages  or  non-verbal  stlaull  such  as  pic- 
tures or  auslc  are  not  discussed.)  Tne  final  section 
addresses  the  problea  of  writing  good  instructions  for 
different  test  purposes. 
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THE  WRITING  TOPIC:  COMTEHT  AND  FORMS 
Section  Over  view 


Testers  select  ani  phrase  the  writing  topic  with 
three  different  concerns  in  aind:  the  need  to  guarantee 
accessible  content,  the  need  to  apecify  or  linit  con- 
tent, and  the  need  to  elicit  particular  kinds  of  writ- 
ing.    The  topics  discussed  below  will  be  evaluated  for 
their  ability  to  guarantee  every  student  aotaething  to ^ 
say,  for  the  Banner  in  which  they  apecify  what  students 
write  about,  and  for  their  appropriateness  in  eliciting 
different  kinds  of  writing  or  writing  for  different 
purposes.     In  the  following  discussion,  rough  distinc- 
tions are  laade  between  four  kinds  of  writing  tasks. 

First,   introspective  or  autobiographical  writing 
is  done  for  purposes  of  exploring  and  sharing  the 
writer^s  personal  world,  and  is  useful  for  Measures  of 
general   fluency.     The  second  kind  of  writing  task  is 
discussed  below  under  "Topics  That  Draw  Upon  Personal 
Eiperience."  Although  testers  are  advised  against  try- 
ing to  distinguish  between  fact  and  fiction  in  student 
narrative,  they  aay  eiaaine  a  student's  narrative  abil- 
ities either  by  appealing  to  the  students*  own  eiperi- 
ences  or  by  appealing  directly  to  their  iaagination, 
asking  thea  to  create  an  imaginary  world,  with  topics 
calling  for  or  allowing  fictional  or  fantasy  writing. 
This  kind  of  task  is  discussed  under  the  heading, 
**Topics  That  Elicit  Made-Up  Stories."*  Soae  of  the 
topics  listed  as  appealing  to  personal  experience  tend 
to  elicit  priaarily  descriptive  as  opposed  to  narrative 
writing.     Other  topics  which  atteapt  to  elicit  purely 
descriptive  writing,  without  reference  to  a  aore  gen- 
eral rhetorical  aia,  are  discussed  in  "The  Writing 
Instructions:     Purposes  and  Probleas,"  in  the  final 
section  of  this  report,  under  the  sub-heading  "The 
Describe  Trap." 

Third,  one  aight  offer  topics  which  call  for  busi- 
ness writing  or  social  writing,  both  done  to  accoaplish 
transactions  typical  of  the  world  outside  of  school. 
Business  writing  topics  such  as  complaint  letters  or 
Job  applications  are  discussed  also  in  the  final  sec- 
tion of  this  report  under  the  sub-heading,  "The  Special 
Conventions  Trap."  Social  or  coaaunity-directed  writing 
includes  arguaents  or  persuasive  writing  about  public 
issues.     Topics  for  assessing  arguaent  skills  or  per- 
suasive writing  are  exaained  below  under  the  sub- 
headings, "The  Probleas  of  Prior  Knowledge  Requirements 
in  Exposition  and  Argument  Topics"  and  "Controversial 
Issues  as  Topics."  Although  writing  is  also  done  in 
school  settings  which  is  aimed  at  persuading  others  to 
agree  with  one's  values,  to  adopt  certain  attitudes,  or 
to  take  certain  actions,  all  such  writing  is  considered 
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social  In  function,  of  Interest  to  testers  when  both 
writing  and  reasoning  skills  are  to  be  measured. 


And  finally,  we  cone  to  expository  writing ,  which 
Is  typical  of  writing  done  In  an  acadealc  setting,  and 
Is  Intended  to  explore  Ideis  and  concepts  about  a  world 
which  Is  shared  by  reader  and  writer,  to  Interpret 
events,  or  to  aake  explicit  the  meaning  of  experiences. 
Expository  writing  Is  also  discussed  In  conjunction 
with  social  or  persuasive  writing,  but  It  differs  from 
the  latter  In  that  the  writers  are  not  asked  to  write 
as  If  they  wish  to  affect  a  reader's  actual  behavior  In 
the  real  world  or  change  the  course  of  events,  but 
rather  as  If  they  wish  to  project  an  Idea  for  examina- 
tion and  evaluation  by  the  reader  as  part  of  a  shared 
process  of  making  sense  of  experience.     These  rather 
abstract  distinctions  between  argument  and  exposition 
may  be  Ignored  for  most  testing  purposes,  since  the 
thinking  and   language  skills  needed   for  these  two  kinds 
of  task  are  often  similar. 

It  should  be  pointed  out  that  not  only  the  topic 
but  also  the  Instructions  to  the  writer  will  determine 
which  of  these  kinds  of  writing  Is  likely  to  be  eli- 
cited.    Introspective  or  autobiographical  writing,  in 
particular,  may  range  from  almost  pure  narration  of 
events  to  almost  pure  commentary,  the  content  of  expo- 
sition.    The   framing  of  the  topic  and  Instructions  can 
encourage  the  writers  In  either  direction — toward  a 
concrete  representation  of  an  experience,  or  toward  a 
reflective  and  explicit  examination  of  the  Ideas 
derived   from  an  experience;  or  they  can  leave  the  writ- 
ers the  full  range  of  possibilities  In  their  own  con- 
struction of  the  task,  as  when  the  test  Is  one  of  gen- 
eral  fluency  rather  than  of  specific  narrative  or  expo- 
sitory skills.     The  remaining  parts  of  this  section 
deal  In  turn  with  specifying  content  for  personal 
experience  writing,  exposition,  and  argument. 

Topics  That  Draw  Upon  Personal  Experience 

Popular  topics  that  provide  the  writer  with  rich 
content  and  elicit  relatively  fluent,  honest  writing 
close  to  the  student's  natural  voice  are  those  which 
encourage  Introspective  or  autobiographical  writing  by 
drawing  upon  personal  experience.     Such  writing  Is  easy 
to  motivate  and  Is  highly  Intentional.     When  someone 
asks  us  w^iat  we  would  wish  for  most  In  the  world,  or 
what  we  recall  that  was  happy  or  sad  or  surprising  from 
our  childhood,  or  whom  we  especially  admire,  we  believe 
the  questioner  to  be  genuinely  curious.     We  recognlxe 
this  rhetorical  situation  from  oral  speech,  and  most  of 
us  welcome  an  Invitation  to  explore  our  own  minds  or 
re-examine  our  experiences  In  conversation  with  a 

o  ^^^^ 

ERIC 


friendly  listener.     Unlike  topic*  which  require  stu- 
dents to  write  about  Inforaatlon  which  they  believe  the 
reader  already  has,  these  topics  allow  students  to 
lapart  Inforoiatlon  which  only  they  possess:  on  what 
they  reaeaber,  what  they  think,  how  they  feel  they  can 
write  with  authority* 

Topics  designed  to  elicit  personal  eipresslon 
rarely  specify  the  precise  subject  of  the  discourse; 
rather,  they  specify  the  predication. 

PROMPT  2a 

NOT 

Write  about   December  25,  1973* 


BUT 


PROMPT  2b 


Write  about  an  eiperlence  you  had  during  a 
faally  holiday  which  was  particularly  aemor« 
able.     It  aay  be  happy  or  sad,  but  should  be 
a  tlffle  when  you  had  strong  feelings. 


Instead  of  naalng  a  specific  event  or  character,  as  one 
■Ight  In  a  test  about  literature,  the  tester  invites 
the  writer  to  select  aaterlal  which  fits  certain 
specifications,  the  aost  Important  usually  being  that 
the  event  or  character  is  aeaorable  in  aoae  way.  The 
underlying  rhetorical  task  la  to  present  the  eiperience 
or  portray  the  character  In  a  way  which  demonstrates 
that  this  person,  object,  or  avent  Is  actually  worth 
writing  about.     Although  the  writer  alght  coaaent, 
eiplaln,  or  analyze,  such  commentary  Is  not  essential 
If  the  writer  Is  sufficiently  skilled  In  descriptive 
and  narrative  techniques  to  make  the  eiperlenbe  memor-* 
able  for  the  reader. 

The  following  topics  have  been  used  to  elicit 
Interesting  and  fluent  Introspective  or  autobiographi- 
cal writing : 

PROMPT  3 

Write  about  a  place  that  Is  special  to  you. 
Help  your  reader  understand  Its  peculiar 
interest  or  sl|nl f Icance  •     The  place  you 
choose  may  be  of  any  klnd««for  eiample,  a 
personal  haunt,  a  historical  spot,  or  an 
institution,  or  a  place  you  remember  vividly 
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fron  early  childhood. 


PROMPT  4 

Write  about  an  object  you  are  especially 
attached  to,  aoaething  that  has  deep  personal 
aeaning  for  you,  sonething  that  has  becoae  a 
part  of  your  life.     You  aight  want  to  con* 
aider  the  way  rou  discovered  it,  the  way  it 
came  into  your  life,  the  way  it  has  taken  on 
aeaning  through  time. 


PROMPT  5 

Te^l  about  something  you  enjoyed  doing  as  a 
child . 


PROMPT  6 

Tell  about  an  experience  from  which  you 
learned  a  lesson. 


PROMPT  7 

Think  of  someone  you  admire  or  resent.  (Hot 
your  parents.)  Write  about  this  person  and 
relate  one  or  two  incidents  that  led  you  to 
adalre  or  resent  this  person,  so  the  reader 
can  understand  why  you  feel   the  way  you  do. 


PROMPT  8 

Describe  a  change  that  has  happened  in  you 
recently  or  an  event  that  happened  a  long 
time  ago  but  still  affects  you.     Tell  of  the 
events  that  surround  it  and  caused  the  change 
in  you  so  your  reader  can  understand  what  is 
now  different  about  you  and  why. 


Students  may  respond  to  these  topics  using  varying  pro- 
portions  of  description,  narrative,  or  commentary. 
Numbers  3  and  4  are  most  likely  to  evoke  descriptive 
writing,  while  number  5  is  more  likely  to  inspire  a 
narrative.     Although  numbers  5  through  8  nay  elicit 
entirely  narrative  accounts,  they  may  also  evoke  dis- 
cussion and  commentary  of  the  kind  that  develops  from 
expository  topics,  as  the  writer  examines  his  subject 
from  a  distance  and  attempts  to  draw  conclusions  or 


reflect  on  the  aeaning  of  his  experience.  The 
researcher's  experience  suggests  that  all  but  the  aore 
advanced  or  nature  writers  will  use  narratives  or  anec- 
dotes whenever  they  are  alio wed  to  io  so.     To  be  sure 
of  eliciting  commentary  and  explicit  discussion  of 
ideas*  the  tester  will  need  topics  that  are  nore  like 
those  used  by  the  College  Board  English  Composition 
Test  (See  also  pp.  56)«  such  as: 

PBOMPT  9 

Each  of  us  assumes  a  personality  to  satisfy 
each  group  into  which  we  enter;  i.e.,  we  do 
not  act  at  home  precisely  as  we  act  with  the 
gang,  in  the  classroom,  or  before  an 
employer . 


This  latter  prompt  models  the  generalizations  that 
form  the  basis  for  expository  writing,  but  it 
encourages  students  to  draw  on  their  own  concrete 
experiences  in  exploring  the  truth  of  the  predications 
advanced  in  relation  to  the  topic.     The  addition  of 
instructions  to  the  writers  helps  translate  the  quota- 
tion into  issues  the  writers  might  recognize  in  their 
own  lives. 

Teachers  find  that  prompts  which  allow  students  to 
write  directly  from  their  own  feelings  and  experiences 
are  excellent  for  assessments  of  general  writing  abili- 
ties, because  such  prompts  point  to  sources  of  meaning- 
ful subject  matter  for  most  writers.     The  wording  of 
the  prompt  jan  be  changed   for  different  age  groups 
without  changing  the  underlying  rhetorical  problem. 
Prompts  3  and  M  above,  for  instance,  could  read  simply: 

PROMPT  10 

drlte  about  yojr  favorite  place. 


PROMPT  11 
Write  about  your  favorite  object. 


Projpts  which  demand  that  students  shift  from 
writing  directly  about  their  own  feelings  and  experi- 
ences to  writing  about  ideas  and  information  have  so 
far  been  used  primarily  with  older  students.  Current 
work  in  cognitive  psychology  suggests  that  students 
below  the  twelve  to  fourteen  year  old  age  range  may 
have  limited  ability  to  sustain  abstract  discourse 
involving  formal  definitions,  reasoning,  analyzing.  On 
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the  other  hand,  young  children  can  often  write  with 
great  involvenent  about  concepts  and  inforaation 
encountered  in  history  and  science  activities,  as  well 
as  writing  about  their  own  values  or  the  nature  of  Cod. 
Using  autobiographical  topics  is  the  best  way  to  test 
for   fluency self«expression,  and  aastery  of  the  writ* 
ten  code,  sinee  personal  topics  provide  the  best 
guarantee  that  the  specified  content  will  be  equally 
accessible  and  interesting  to  all  writers. 

Cuin|  Personal  Topics  for  Content .  Choosing 
interesting  topi cs  and  expr essing^thea  siaply  will 
ensure  that  aost  students  can  find  soaething  Meaningful 
to  say  in  their  writing.     But  soae  of  the  above  topics 
include  content  specifications  which  **cue**  students  to 
see  the  full  potential  of  the  topic,  helping  thea 
recognize  relevant  aaterial  froa  their  own  lives,  and 
clarifying  the  tester's  expectations.     In  the  following 
autobiographical  proapt,  the  ouing  is  underlined: 

PROMPT  12a 

Think  of  an  occasion  when  you  experienced 
soaething  for  the  first  tiae.  aay  be 

soaethi ng  you  later  caae  to  take  for  granted , 
but  for  soae  reason ,  the  Ti  rs t  tiae  was 
aeaor able .     Write  about  your  experience  so 
that  your  reader  understands  your  feelings  at 
the  tiae  and  why  i t  stays  in  your  aeaor y . 


This  prompt  has  elicited  interesting  writing  froa  high 
school  students  ranging  froa  pure  narrative  to  inforaal 
expo  si  tion . * 

In  another  assessaent,  the  prompt  was  aore  ela- 
borately cued.     (Woodworth  and  Keech,   1980.)  Testers 
aay  create  however  aany  cues  they  think  will  be 
relevant  to  their  students*  experiences: 

PROMPT  12b 

Think  of  an  occasion  when  you  experienced 
soaethi  ng  for  the  first  tiae. 

CUES: 

There  is  a  book  called  The  First  Tiae ,  in 
which  famous  people  describe  their  first  kiss 
or  first  love  affair.     But  there  are  aany 
other  iaportant  First  Tiaes  in  our  lives. 


See  Chapter  2. 

1  n  O 
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fpoii  the  first  tiae  we  titd  our  own  shots,  or 
wtnt  to  school  or  camp,  to  our  first  time  In 
deep  water,  or  first  big  fame  or  first  speech 
In  front  of  a  (roup.     Some  of  these  occasions 
are  first  and  only  times.     Others  are  the 
first  of  many  times,  but  for  some  reason  the 
first  time  was  memorable. 

INSTRUCTIONS: 

Choose  one  of  these  occasions  that  stands  out 
In  memory  and  write  about  It  so  that  your 
reader  can  understand  your  experience  and  why 
It  WIS  memorable. 


Think  of  an  occasion  when  you  experienced 
soaethlnf  for  the  FIRST  TIME,     It  may  have 
been  something  you  later  came  to  do  more 
easily  or  something  you  now  take  for  granted; 
but,  for  some  reason,  your  first  time  was 
memorable.     Imagine  that  you  are  writing  to 
someone  who  has  Just  had  a  similar  experi- 
ence, OR  to  someone  who  Is  about  to  have  such 
an  experience*     Tour  writing  might  help 
prepare  your  reader  for  the  experience,  or  It 
might  help  your  reader  understand  that  other 
people  have  gone  through  the  same  kind  of 
thing. 


The  process  of  cuing  substitutes  for  class  discus 
slons  which  enable  students  to  think  of  experiences  In 
their  own  lives  they  might  wish  to  write  about.  The 
cuing  may  clarify  the  kind  of  experience  the  tester 
expects  the  student  to  write  about,  explain  the 
writer^s  options,  or  simply  stimulate  ideas  as  in  the 
fol lowing  prompts : 


Compare  two  stores  you  know  which  sell  simi- 
lar items.     Consider  such  things  as  prices, 
mere  hand  ise ,  service  ,  atmosphere ,  location, 
or  other  special  features,     tfnlch  of  the  two 
stores  do  you  prefer?  Explain  why. 


Compare  two  actual  holidays  or  special  events 
you  celebrate  In  your  family  (such  as 
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PROMPT  13 


PROMPT  14 
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Thanksgiving,  Passovtr,  Christaas,  tht  Fourth 
of  July,  your  birthday,  ttc)  Consider  such 
things  as  food,  decorations,  activities, 
behaviors,  ataosphere,  or  other  special 
features.     «nich  of  the  two  do  you  prefer? 
Es plain  why. 


Elaborate  cuing  has  two  dangers,  however.  First, 
poor  readers  aay  be  discouraged  by  the  sheer  aaount  of 
text  in  the  writing  proapt,  even  to  the  point  of  not 
writing,  or  writing  off-topic.     Second,  unless  tha 
cuing  is  carefully  phrased,  it  aay  seea  to  the  student 
to  offer  coaaands  rather  than  suggestions.     The  follow- 
ing proapt,  recoaaended  by  a  state  agency  for  use  in 
eleaentary  schools  (California  State  Board  of  Education 
Handbook  on  Writing  AssessaJfft,   1979)  includes  highly 
directive  cuing,  intended  to  ensure  that  students  write 
at  soae  length  and   in  coaparable  ways,  but  it  iaposes 
strong  liaits  on  what  students  can  choose  to  write 
about : 

PROMPT  15a 

Tou  have  aet  a  aan  froa  outer  space  who  has 
landed  on  earth  near  your  school  playground . 
He  can  understand  English,  but  he  does  not 
know  anything  about  school  here  on  earth. 
Describe  your  school  for  hia.     Tell  hia  about 
your  school  building.     Tell  how~the  school 
building  looks  on  the  outside  and  the  iii?lde. 
Tel  1  hia  about  your  teacher  and  your  class- 
aates.     (italics  added) 


The  proapt  has  several  unfortunate  characteris- 
tics.    Using  a  spice  aan  as  audience  serves  no  purpose 
that  would  not  be  better  served  by  a  huaan  audience, 
soaeone  who  had  a  believable  reason  to  be  interested  in 
a  written  description  of  the  school.     More  sericus 
probleas,  however,  occur  in  the  cuing  for  content, 
which  gives  students  too  auch  to  do,  preventing  thea 
froa  focusing  and  writing  with  acre  depth  about  one  or 
two  aspects  of  the  school.     (See  discussion  below  on 
•Focusing".)  The  cuing  eaphasites  the  physical  build- 
ing, which  is  perhaps  the  least  interesting  and  aost 
difficult  thing  to  write  about.     (See  discussion  below 
of  the  -Describe  Trap-.)  Finally,  the  iaperative  tone 
of  the  proapt  allows  the  students  no  freedoa  to  choose 
which  parts  of  the  school  are  significant  to  thea  and 
which  they  would  like  to  write  about.     The  top-scoring 
student  papers  that  were  offere-  as  exaaples  were 
extreaely  liaited  in  what  they  atteapted  and  generally 
lacked  focus  and  coherence.     The  following  alternative 
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Is 


suggested : 


PROMPT  15b 


Imagine  a  ffrlend   of  yours  Is  about  to  aove  to 
town.     Your   frlend^s  naae  is  Pat.     Pat  will 
be  going  to  your  school.     Below,  write  a 
letter  to  Pat,  describing  one  or  two  things 
you  think  a  new  student  should  know.  You 
might  tell  about  what  you  do  at  lunch  tine, 
or  describe  the  games  y4u  play.     You  night 
describe  your  favorite  part  of  the  classroom 
or  what  your  teacher  allows  you  to  do  In 
class  activities. 


This  version  has  not  been  pilot^tested ,  so  It  can- 
not be  asserted  that  It  is  aore  successful  than  the 
former  version  In  eliciting  Interesting  writing.  On 
the  other  hand,  this  wording  represents  a  more  nacural 
rhetorical  framing  of  the  task,  putting  the  student  In 
the  role  of  Intentional  communicator  rather  than  that 
of  direction-follower.     Because  the  framing  of  a  test 
question  often  becomes  a  model   for  the  teacher^s  fram- 
ing of  a  classroom  writing  assignment,  the  test  prompt 
should  represent,  as  closely  as  possible,  the  kinds  of 
rhetorical  situations  In  which  students  are  encouraged 
to  shape  their  own  discourse  meaningfully.     Wnere  stu- 
dents have  had  little  practice  In  writing,       tester  may 
find  they  write  longer  papers  In  response  to  prompts 
such  as   15a  above  which  lists  a  set  of  things  to  men- 
tion.    But  the  aore  naturalistic  framing  of  Prompt  15b 
could   well  provide  students  with  genuine  opportunities 
to  wr I te  . 

The  following  prompt,  fro«  a  California  elementary 
school   assessment,  raises  useful  guiding  questions 
about  what  students  should  Include  and  how  their  essays 
might  be  shaped,  but  It  had  an   unfortunate  side  effect 
on  some  Inexperienced  writers: 


Write  about  your   favorite   place.     Name  the 
place  and  tell  where  It  la.     Tell  how  often 
you  go  there.     Why  is  this  place  special  to 
you?     How  do  you  feel  when  you  are  there? 
What  are  some  sights,  sounds,  colors  and 
smells  In  your  favorite  place? 


All  the  questions  In  this  prompt  provide  excellent  ways 
to  stimulate  Ideas  during  brainstorming  or  prewrltlng 
sessions.     In  fact,  the  cues  In  this  prompt  were  used 
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for  a  pretfriting  exercise  on  a  sheet  of  paper  wyiich 
listed  eash  coataand  or  question  followed  by  blank 
spaces  for  the  student^s  responseo     During  the  prewrit** 
ing ,  and  again  on  the  test  day,  some  students  perceived 
the  writing  test  as  a  short-answer  quiz  and  wrote  words 
and  phrases  rather  than  sentences  in  response  to  the 
eosaands  or  questions.     To  aany  students,  the  following 
type  of  response  seens  acceptable,  given  the  phrasing 
of  the  prompt : 


WRITING  SAMPLE 


Disneyland.     Hollywood.     Once  a  year. 

Because  it  is  fun.     I  feel  good.  Happy. 

Tired.  Sights:     Mickey  Mo  use,  rides,  African 

house.  Sounds:     loud  ro«rs,  lots  of  people. 
Colors:     red,  yellow,  blue,  all  colors. 
Smells:     cotton  candy,  caramel  apples,  animal 

smells,  people  smells. 


It   is  important  then  to  make  the  prompt  sound  less 
like  a  set  of  questions  to  answer  and  more  like  an 
invitation  for  communication,  as  in  this  revision: 


Everyone  has  a  special  place  where  they  like 
to  go.     Write  about  your  favorite  place  to 
go.     Describe  the  place  so  your  reader  can 
imagine  what  it  is  really  like.     Tell  about 
what  you  do  there,  and  how  you  feel  when  you 
are  there.     Help  your  reader  see  why  this 
place  is  special. 


In   summary ,  when  testers  desire  to  give  students 
assistance  in   finding  and  selecting  personally  meaning- 
ful content,   they  should  consider  methods  of  cuing 
which  avoid  sounding  too  much  like  directions  to  the 
writer  and  more  like  genuine  options.     Cues  should  be 
arranged  if  possible  to  suggest  a  natural  way  of  plan- 
ning the  response  to  the  topic.     Elaborate  cuing  is 
appropriate  only  if  students  ire  competent  readers  and 
have  adequate  time  to  consider  the  suggestions  bafore 
beginning  to  write.     Elaborate  cuing  removes  some  of 
the  work  of  composing  for  poor  writers  but  may  hinder 
good  writers.     In  fact,  the  best  writers  will  often 
regard  the  listed  options  as  things  not  to  write  about, 
thinking  that  "everyone  else  will  write  about  those,** 
and  seeking  instead  to   find  a  subject  more  novel  than 
those  suggested  by  the  tester.     For  tests  of  general 
writing  ability  which  use  personal  experience  topics,  a 
well-phrased   topic  and  simple  cuing  are  usually  ade- 
quate to  stimulate  content  in  the  resultant  writing. 
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Topics  Thit  Elicit  Made-Up^  Stories 

Literary  writing  Is  the  wrltlnj  of  fiction,  auto- 
biography, and   poetry.     Some  children  will  begin  to  use 
literary  techniques  quite  early  in  responding  to  the 
topics  listed  under  personal  writing,  above.     On  the 
other  hand,  if  it  is  story-telling  abilities  per  se 
that  are  being  tested  for,  prompts  that  draw  on  writ- 
ers'  direct  experience  would  have  to  be  replaced  by 
prompts  that  challenge  their  laag in  a t ion S  ,  calling  for 
fantasy  or  fiction  stories. 

Frequently  stories  are  elicited  by  prompts  which 
give  starter  lines  or  conclusions.     A  middle  school 
Included  the  following  task  as  one  part  of  its  writing 
assessmen  t : 

PROMPT  17 

rfrlte  a  story  that  begins:      'One  day  I  found 
a  mag Ic  mac  hi  ne . * 

Other   successful   starters  have  been: 

PROMPT  18 

This  morning  on  my  way  to  school,   I  saw  a  big 
black  hole. 

PROMPT  19 

The  last  thing   I  remember  was  the  ship  in 
flames,  people  scream  Ing  and  running  for 
lifeboats.     Then  I  blacked  out.     When   I  came 
to  ,   I  wa  s  .  •  . 


The  instruction  part  of  these  prompts  should 
specify  that  the  examiner   is  calling  for  the  creation 
of  a  narrative  so  that  the  atudent^a  sense  of  story 
will  give  structure  to  the  piece.     In  these  pieces, 
flashes  of  personal  insight  or  memories  or  real  feel- 
ings that  arouse  the  reader's  sympathy  will  be  less 
Important  than  story  structure — imaginative  incidents, 
vivid  details,  conflict,  pacing,  coherence,  good  clo- 
sure.    Story-writing   is  a  special  ability  and  uence 
requires  a  specialized   test.     There  is  no  reason  not  to 
score  such  a  test  hoi  1 s 1 1 c a  1 1 y  .  but  one  cannot  use  such 
test  topics  as  if  they  were  parallel   in  task  demands  to 
the  autobiographical   topics.     The  topics  offered  to 
ins;^ire  writing  about  personal  experience,  while  they 
often  elicit  literary  strategies,  may  also  encourage 
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Pretend  I  a«  the  Wl3h  Fairy  wno  can  aake  your 
wishes  coae  true.  I  want  you  to  write  ae  a 
letter  to  tell  me  what  you  wish  for.  Think 
of  several  things  you  wish  for.  (pause)  How 
then,  choose  the  one  thing  you  wish  for  the 
very  aost . 

Tell  me  what  it  is  by  finishing  the  sentence, 
«I  wish  I  had..."  Wnat  do  you  wish  you  had? 
After  you  write  down  its  naae,  then  go  on 
writing  and  tell  ae  all  about  it.     Wnat  is  it 
like?     Write  down  everything  you  think  and 
feel   about  your  wish.* 

Tne   instructions  above  can  easily  be  aodified  for 
parallel  versions  of  the  "wish"  prompt: 

PROMPT  20b 

I  wi  sh  I  wer e . . . 
I  wi  sh  I  could . . . 
I  wi  sh  I  knew. . . 


In  the  early  years  of  holistic  assessment  at  a  Bay 
Area  high  school,  the   teacher  test-makers  were  pri- 
marily concerned  about  creating  prompts  that  students 
would  enjoy  writing  about  In  the  annual  assessment.  At 
that  time  they  were  not  using  the  annual  assessment  for 
placement  purposes  or  even  program  e^^aluatlon.  The 
following  parallel  prompts,  used  In  their  first  three 
annual   testings,  were  felt  to  be  highly  successful  In 
eliciting  writing  that  was  both  competent  and  Interest- 
ing : 

PROMPT  21 

Mrlte  about  an  event  you  wish  you  had  wit- 
nessed or  could  witness.     The  event  can  be 
real  or  Imagined;   the  time  of  the  event  can 
be  past,  present,  or  future.     Make  it  clear 
why  the  event  Is  significant  to  you.  (1973) 


T^fne  advantage  of  oral  Instructions  In  elementary  or 
even  Junior  high  Is  that  guidance  and  prewrltlng  stimu- 
lation is  provided  without  transforming  the  writing 
test  into  a  readi^ng  comprehension  test.  In  holistic 
assessments  administered  In  more  than  one  class,  howev- 
er, teachers  should  read  a  standardized  set  of  oral  In- 
structions to  avoid  variations  in  the  task  descrip- 
tion.) 
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use  of  language  patterns  that  are  similar  to  those  of 
oral  language.     Hence  the  student  may  attend  more  to 
the  problems  of  transcribing  these  ideas  "aloud"  in 
writing,  than  to  coping  with  the  additional  constraints 
of  trying  to  create  a  made^-up  story. 

Making  up  a  story  is  possibly  an  easier  task  for 
children  at  some  stages  of  development  than  it  writing 
about  their  feelings,  wishes,  or  experiences.  For 
younger  writers,  writing  means  writing  stories.  They 
may  not  make  up  very  good  stories,  or  recognize  when 
they  have  a  beginning,  middle,  or  end,  but  they  some- 
times write  a  great  deal  more  in  the  attempt  than  if 
the  tester  had  asked  them,  for  instance,  to  describe  a 
favorite  object.     It  may  be  that  they  understand  the 
rhetorical  Intent  behind  the  command,  "write  a  story," 
whereas  they  do  not  recognize  the  rhetorical  Intent 
behind  the  command,  "tell  me  about  someone  you  admire." 
On  the  other  hand,  one  elementary  school  teacher  points 
out  that  students  need   far  more  time  and  space  to - 
develop  a  story.     They  often  write  many  pages  which 
they  are  reluctant  to  re-read  or  revise.  (Gentry, 
1980. ) 

Starter  lines  or  conclusions  are  not  effective  for 
use  with  older  students.     The  more  experienced  writers 
often  rebel   at  Intervention  In  their  creative 
processes.     Requests  to  write  out  of  personal  experi- 
ence or  to  write  about  Imaginary  situations  are  more 
likely  to  be  honored  as  legitimate  prompts.     A  good 
exa-nple  of  such  a  topic  Is  the  job  application  which  Is 
discussed  below.   In  the  last  section  under  "The  Special 
Conventions  Trap". 

Hypothetical  Situations  as  Subjects  for  Writing 

Many  teachers  find  that  topics  which  ask  students 
to  imagine,   pretend,  or  wish  often  Inspire  lively  and 
Interesting  writing.     An  effective  elementary  school 
test  consists  of  the  following  words  on  a  piece  of 
blank  paper  : 


PROMPT  20a 


Dear  Wish  Fairy, 
I  wish  I  had 


Tne  teacher  gives  the   following  Instructions  orally: 


erIc  ' 


If  you  had  to  choose  to  be  something  other 
than  a  human  being,  what  plant  or  animal  or 
other  fora  would  you  choose?  In  your  writ- 
ing, give  your  reader  some  idea  of  what  you 
think  it  would  be  like  to  be  that  form,  and 
of  why  you  chose  it.  (197U) 


PROMPT  23 

If  you  could  change  places  with  someone  else, 
wno  would   it  be?     Tne  person  you  write  about 
can  be  living,  dead,  drawn   from  past  or 
jiresent,   from  books,  films,  etc.,   or  from 
your  own   imagination.     In   your   writing  give 
your  reader  some  idea  of  what  it  would  be 
like  to  be  that  person,  and  of  why  that  life 
appeals  to  you.  (1975) 


However,   these  three  topics  require  writers  to 
project  themselves  into  hypothetical   situations.  A 
major  problem  with  these  hypothetical  topics  is  that 
this  led  many  students  to  begin   their  responses  using 
conditional  verb  forms,  such  as  "I  would...,"  "I 
could...."  Only  the  very  best  high  school  writers  who 
began   in  this  way  were  able  to  maintain  a  consistent 
point  of  view  throughout  their  essays.     Robinson  (1931) 
and  Kirrie  (1930)  have  both  warned  against  the  use  of 
hypothetical   situations  for  testing  writing  skills 
because  of  the  need   for  students  to  use  conditional  or 
subjunctive  verb  phrases.     As   Robinson  says,  "When  stu- 
dents are  put   Into  situations  where  they  have  to  use 
the  conditional,  they  invariably  demonstrate  that  they 
cannot . " 

However,   each  of  three  prompts  listed   above  (21 
22,   23)   ended  with  these  Instructions: 


You  may  do  this  writing  as  a  journal  entry, 
character  sketch,  dialogue,  letter,  story, 
autobiographical  essay,  argument,  poem,  or 

othar  form. 


Even  though  many  testers  advise  against  offering  stu- 
dents such  a  wide  range  of  choices  (see   the  discussion 
of  the  "Plck-A-Form  Trap^   that  follows),   these  alterna- 
tive modes  often  provide  students  with  a  way  of  avoid- 
ing what  Robinson  calls  the  "Great  Conditional  Problem" 
waen  they  are  confronted  with  hypothetical  situations. 
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Many  students  in  the  assessnents  where  the  hypothetical 
tcpics  were  used  avoided  the  conditional  altogether  by 
olunaing  directly  into  narratives  or  by  using  purely 
descriptive  writing.     For  the  prompt  that  asked  the«  to 
Imagine  being  non-human,  some  students  adopted  the 
voice  and  persona  of  the  non-human  being  and  wrote  a 
kind  of  interior  monologue,  e.g.,  a  beetle  reflecting 
on  "my  life  as  a  beetle." 

The  test's  purpose  must  determine  whether  to  use 
imaginary  situations  as  writing  prompts.     The  experts 
cited  advise  against  using  imaginary  situations  as 
prompts;  however,  if  one  wishes  to  test  the  student  s 
ability  to  sustain  the  conditional  point  of  view, 
prompts  such  as  the  following  are  useful: 

PROMPT  24 

What  would  you  do  if  you  had  a  million  dollars? 

The  following  prompts  describe  iaajinary  situa- 
tions which  proved  interesting  to  student  writers,  in  a 
project  designed  to  pilot  test  a  number  of  topics.* 
They  elicited  mostly  narrative  and  descriptive  writing 
free  of  the  conditional  problem  because  these  prompts 
placed  students  inside  the  situation  rather  than  asking 
them  to  comment  on  what  would,  should,  or  might  happen 
might  happen  if  such  a  situation  occurred: 

PROMPT  25 

Hnile  visiting  in  your  grandmother's  house 
you  find  a  large  trunk.     Propped  on  a  chair 
beside  the  trunk  Is  a  card;  on  It  In  your 
grandmother's  handwriting  are  the  words:  "To 
my  only  grandchild:     Inside  this  trunk  you 
will  find  what  you  have  always  wanted," 
Describe  the  gift  and  how  you  will  use  It. 

PROMPT  26 

Imagine  that  you  are  trapped  In  one  of  the 
following  situations  and  you  cannot  escape 
for  several  hours.     Tell  what  you  do  to 
escape  or  to  pass  the  time  until  you  are  res- 
cued : 


Ts^ntTTriTiTCqunt/  Writing  Assessment  Field  Test  of 
Prompts  In  chapter  3. 

1  ^  ■ 
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A. 


an  tlevator 
an  abandoned  alnt 
a  locktd  rooa 
a  life-boat 


PKOAPT  27 


laaglne  that  you  havt  just  landed  on  the 
moon.     Write  about  what  happens  after  you 
cliab  out  of  your  space  ship  and  set  foot  on 
the  surface  of  the  soon. 


The  conditional  problem  does  not  seers  to  haaper 
younger  writers  who  usually  have  strong  feelings  about 
fairness  and  a  good  sense  of  what  they  might  do  in  an 
imaginary  situation.     Thus  they  often  respond 
enthusiastically  to  questions  that  begin.  "What  should 
you  do  when...."  The  following  exercise  has  been  used 
with  fifth  and  sixth  graders  as  an  informal  classroom 
activity. 


Pretend  you  write  the  advice  column  in  our 
school  newspaper.     You  must  answer  the  fol- 
lowing letter: 

Dear   Happy  Helper: 

I  have  a  little  sister  who  always  hits 
me  when  I  have  a  toy  or  a  cookie  she  wants. 
Hy  grandmother  says   I  can't  hit  her  back. 
She  says  I  should  share,  but   I  get  tired  of 
sharing.     What  do  you  think  I  should  do? 


The  research  finds  that  responses  to  this  prompt 
will  generally  consist  of  conditional  forms  such  as. 
"You  should...."   "You  might..."  or  imperatives,  such 
as.  "Do  this."  Thus,  students  benefit  from  prewriting 
discussions  during  which  they  can  practice  giving  rea- 
sons for   their  advice.     Wnile  other  prompts  are  better 
tests  of  narrative  abilities,  this  one  tests  the  abil- 
ity to  make  suggestions  or  give  advice. 


PROMPT  28 


Signed  . 


In  Trouble 
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Imaginary  situations  that  present  moral  dileiaoas 
("tfhat  would  you  do  if...**)  can  become  loaded  questions 
if  the  teacher  readers  actually  believe  that  there  is 
really  only  one  rifht  answer.     For  instance*  in  the 
above  prompt,  if  readers  accepted  only  non-viol«nt 
solutions  to  the  problem,  students  who  made  other 
augsestions  would  be  scored  dQwn.     If  students  were  not 
told  in   advance  that  there  was  only  one  correct  kind  of 
response,   thi9  would  become  a  ''loaded  question.** 

In   scoring  writing  from  prompts  such  as  the  above 
which  involve  moral  dilemmas  or  value  choices,  taacher 
readers  naturally  tend  to  Judge  some  answers  as  more 
bright"   or  more  admirable  than  others.     However,  in 
holistic  assessments  of  general  writing  abilitiest 
evaluators  probably  desire  to  Judge  whether  the  stu- 
dents can  communicate  clearly  what  they  think  a  person 
should  do  in  a  given  situation,  not  whether  their 
choice  seems  "good"  or  "bad."  Thus  trainers  generally 
try  to  help  raaders  resist  letting  moral  Judgments 
strongly  influence  scores— unl ess  such  Judgments  are 
included  in  the  goals  of  the  assessment. 

Problems  of  t»r ior  Knowledge  Requirements  in 
Exposition  and  Argument  Topics 

For  axposition  and  argument  the  question  of  par- 
ticular prior  knowledge  becomes  more  involved  in  the 
writing  task.     The  better  informed  writer,  the  writer 
who  has  thought  about  an  issue,  is  likely  to  produce 
more  cogent  arguments,  more  convincing  examples  to  sup- 
port his  or  her  position  or  to  explain  his  or  her 
thesis.     Some  writing  tasks  have  failed  to  elicit  good 
writing  even  from  good  students  when  none  of  the  stu- 
dents tested  had  adequate  information  about  the  topic 
presented  for  discussion,  or  when  too  many  of  the  stu- 
dents had  not  previously  considered  a  problem  in  suffi- 
cient depth  to  write  any  but  the  most  simplistic  and 
unreflective  comments.     The  following  topic  was  elim- 
inated from  one  assessment  after  teachers  tested  it  by 
attempting  to  write  on  it  themselves: 

PROMPT  29 

Explain  your  agreement  or  disagreement  with 
the   following  statement:  The  history  of  women 
in  our  couritry  is  the  history  of  oppression. 


Two  teashers  admitted  that  they  did  not  know  enough 
about  the  history  of  women  to  write  a  sound  essay  on 
the  subject.     The  topic  would  be  appropriate,  of 
course*   if  all  students  being  tested  had  Just  completed 
a  course  on  the  history  of  women  in  their  country.  In 
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that  ease  the  topic  would  test  course  content  mastery 
In   addition  to  writing  ability;  teacher  readers  still 
would  need  special   training  to  reach  agreement  on  what 
constituted  an  adequate  response  to  the  topic. 

In  one  large  asses sment  that  was  ached uled  to  be 
held  o^er  two  years  beginning  during  the  drought  In 
California,   testers  pllot«tested  a  topic   that  asked 
students  to  describe  the  effects  that  drought  had  on 
the  campaign  urging  people  to  conserve  water,   and  to 
make  specific  recommendations  about  bow  to  do  so.  Stu- 
dent responses  were  well-Informed  and  Imaginative, 
often  passionate,  but  the  topic  had  to  be  dl^opped 
because  testers  could  not  be  sure  that  the  second  year 
of  the  assessment  would  provide  a  drought  or  a  parallel 
crisis  as  certain   to  have  touched  the  lives  of  all  stu- 
dents In  the  sample.* 

To  some  extent  the  expository  or  argument  topic 
which  requires  students  to  marshal  evidence  from  the 
realtor  literary  world  Is  a  test  of  general  knowledge. 
It  becomes  a   test  of  special  knowledge  only  if  content 
is  narrowly  specified,  as  it  must  often  be  to  shape  a 
persuasive  topic.     A  test  of  general  writing  ability 
need  not  discriminate  in  favor  of  students  whose  spe- 
cial interests  or  background  give  them  access  to  infor- 
mation not  widely  shared,   if  the  topic  allows  students 
to  argue  directly  from  their  individual  experiences 
and/or  reading.     Better  informed  students  will  have  an 
advantage  in  any  case,  of  course,  but  the  choice  of 
topic  can  minimize  or  maximize  that  advantage,  depend- 
ing on  the  purpose  of  the  assessment:  whether  the  writ- 
ing task  is  intended  primarily  as  a  skills  test  or  as  a 
special  content  test. 

To  avoid  problems  of  special  knowledge  require- 
ments,  testers  can  provide  a  body  of  relevant  facts  or 
a  literary  passage  to  be  commented  on  or  argued  from. 
Writing  from  given  information  may  test  reading, 
analysis,  and  reasoning,  as  well  as  writing  abilities, 
but  no  student  is  put  at  a  disadvantage  for  not  having 
encountered  specific  facts  or  read  specific  litera- 
ture.* (See  Prompt   31a)  Another  option  is  to  provide  a 

(*Note:  Carnegie  funded  evaluation  of  Bay  Area  Writing 
Project ,  NIC  hail  Scriven ,  Director ,  1979. ) 
*This  kind  of  prompt  is  used  in  the  military  and  busi- 
ness worlds  on  writing  tests  for  Job  advancement,  and 
resembles  performance  measures  called  "In -basket 
tests",  where  candidates  are  asked  to  respond  to  the 
kind  of  problems  likely  to  appear  in  their  "In- 
baskets."  This  is  also  a  common  format  for  college 
placement  tests  like  the  Subject  A  exam  at  University 
of  California,  Berkeley     which     provides     an     essay  or 
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stimulus — quote,  word,  picturt,  idea,  etc,— and  ask 
students  to  draw  on  their  own  knowledge  op  eiperience 
to  shape  their  essays.     Testers  aust  give  clear 
instructions  to  students  to  draw  on  their  reading  and 
experience  for  examples  to  explain  their  theses.  Tnis 
is  the  coaaon  foraat  on  aany  college  placeaent  exaas. 
(See  pp.   Proapts  3*  and  35  below.)  Again,  test-aakers 
may  provide  a  rhetorical   situation  sufficiently  coaaon 
that  every  student  can  iaagin^  facing  such  a  task  and 
allow  students  a  range  of  alternatives,  including  iaa- 
ginary  ones,  in  their  solution  to  the  problea.     A  good 
exaapla  of  such  a  topic  is  the  Job  application  situa- 
tion presented  in  Proapt  43* 

Controversial  Issues  As  Topics 
for  Persuasive  Writing"^ 

Controversial  issues,  such  as  abortion  or  capital 
punishaent,  seea  to  provide  especially  appropriate 
topics  for  tests  of  persuasive  writing.     Students  can 
be  expacted  to  have  auch  to  say  about  either  side  of  a 
current  controversy;  however,  their  papers  aay  reflect 
unconsidered  parroting  of  opinions  heard  in  the  aedia 
or  froa  parents.     Controversial  topics  can  create  spe- 
cial probleas  for  holistic  assessors  if  the  subject 
arouses  strong  biases  in  both  student-writers  and 
teacher-readers  and  distracts  readers  froa  asking  a 
careful   evaluation  of  the  student's  writing  abilities. 

Topics  based  on  issues  of  sexisa,  for  instance, 
often  inspira  heated  and  honest  writing,  but  they  also 
aay  lead  to  expressions  of  hostile,  sexist  views  on  the 
part  of  both  aales  and  feaales.     Such  views  can  anta- 
gonize teacher-readers  who  feel  equally  strongly  about 
the  issues.     Special  training  can  overcoae  these 
responses  to  soae  extent,  but  for  aost  test  purposes, 
it  aay  be  wise  to  avoid  direct  stateaents  of  extreae 
positions,  such  as  the  following: 

PROMPT  30a 

Men  are  superior  to  woaen. 


PROMPT  30b 
Woaen  are  superior  to  aen . 


literary  passage  for  critical  analysis,  thus  testing 
reading  and  analytic  skills  to  be  taught  in  Freshaan 
English  courses. 
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INSTRUCTIONS: 


Agree  or  disagree  with  the  stateaent,  giving 
reasons  for  your  opinion. 


A  different,  slightly  aore  sophisticated,  phrasing 
appeals  less  to  prejudice  and  aore  to  observations  and 
tiperiences,  thereby  inspiring  aore  interesting  and 
acorable  writing  (Caplan  and  Keech,  1980): 

PROMPT  30c 

Woaen  are  better  off  than  aen  in  today's 
so  c  itt y  • 


INSTRUCTIONS: 


Agree  or  disagree  with  the  stateaent,  giving 
reasons  for  your  opinion. 


An  entirely  different  approach  to  this  topic  was 
taken  by  an  evaluator  who  wished  to  test  the  students' 
understanding  of  logic   and  their  ability  to  recognize 
good  and  poor  arguaents  (Keech,  1978): 

PROMPT  30 d 

INSTRUCTIONS: 

Explain  what  is  wrong  with  the  following 
argument:  ••It's  pretty  clear  that  on  the 
whole,  woaen  are  superior  to  aen.     In  the 
physical  diaension,  the  best  evidence  for 
this  is  aiaply  that  woaen  live  longer ,  and  by 
a  substantial  number  of  years.     Other  evi- 
dence is  that  woaen  are  virtually  laaune  to 
several  diseases  that  are  fatal  for  aany  aen; 
and  on  the  aental  side,  there  la  the  siaple 
fact  that  girls,  on  the  average,  do  better  at 
school  than  boys." 


ERIC 


Because  the  instructions  preceding  the  argument  called 
for  criticism  of  the  position  taken,  students  under- 
stood that  they  were  to  find  something  wrong  with  the 
argument.     Many  students  simply  attacked  the  basic 
assertion  that  ••Women  are  superior  to  men,"  as  if  the 
question  had  been  worded  that  way.     A  large  number, 
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however,  responded  to  the  test-aaker»s  Intention  and 
attacked  the  prealses  of  the  argument,  as  well  as 
agreeing  or  disagreeing  with  the  central  assertion  and 
offering  soae  defense  of  their  own  views.  Scoring 
these  papers  Involved  the  tester  and  the  readers  In 
soae  controversy  over  whether  students  who  addressed 
the  assertion  without  attacking  the  argument's  faulty 
premises  had  adequately  addressed  the  test  question. 
Ultimately,  readers  Insisted  on  giving  full  credit  to 
students  who  Interpreted  the  Instructions  as  an  Invita- 
tion to  quarrel  with  the  position  stated,  as  well  as  to 
those  who  pointed  out  deficiencies  In  the  form  of  the 
argument,  since  the  former  set  of  papers  Included  some 
of  the  best  writing  In  the  sample.     Actually,  the 
disagreement  between  test-maker  and  readers  reflected  a 
deeper  disagreement  about  the  purpose  of  the  assess- 
ment: whether  the  measure  wa3  to  be  a  an  overall  test 
of  self-expression  or  a  more  precise  test  of  certain 
reasoning  abilities. 

The  scoring  of  this  test  question  could  be  based 
on  whichever  of  the  two  purposes  were  paramount,  but  in 
either  case.  It  revealed  both  the  dangers  and  the 
advantages  of  using  highly  controversial  topics.  All 
students  wrote  fairly  fluently  to  the  topic;  many  wrote 
with  passion  and  conviction,  drawing  on  personal 
experience,  assessing  personal  values,  qualifying 
definitions  (of  ••super  lor** ,  for  Instance)  or  discussing 
the  practical  Implications  of  taking  the  given  posi- 
tion.    Even  poor  writers  generally  made  one  or  more 
cogent  points.     On  the  other  hand,  some  students  Inev- 
itably expressed  views  that  offended  some  readers.  One 
boy  was  scored  down  by  two  readers  (one  male,  one 
female)  because  of  the  extremism  of  his  views. 
Although  he  had  Intelligently  attacked  the  faulty  prem- 
ises of  the  given  argument,  he  had  frankly  admitted  to 
biases  that  the  readers  thought  outrageous  and  offen- 
slve«     In  a  test  of  general  writing  ability,  his  low 
score  was  misleading,  because  he  had  written  cleverly 
and  effectively. 

Here  the  hollatlo  nature  of  the  Judgment  breaks 
clown,  when  the  readers  are  to  reconcile  the  quality  of 
the  writing  with  the  presumed  wrongheadedness  of  the 
Ideas  expressed,  and  they  feel  compelled  to  Judge  one 
to  the  exclusion  of  the  other.     Such  papers  often 
receive  discrepant  scores  In   their  two  readings. 

Controversial  Issues  can  be  presented  interest- 
ingly as  legitimate  conflicts  of  Interest,  as  in  the 
following  prompt  which  was  designed  to  test  powers  of 
observation,  reasoning,  and  argument  through  a  per- 
suasive writing  task  administered  to  a  group  of  enter- 
ing freshmen  in  a  large  state  university  (Cooper,  et 
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al..  1979). 


PROMPT  3la 

At  the  place  «here  you  work,  a  woaan  has  just 
quit  her  job,  leaving  vacant  the  company's 
only  executive  position  ever  held  by  a 
feaale.     The  Board  of  Directors  has  stated 
their  preference  that  a  woaan  replace  her  In 
order  to  fulfill  an  Afflraatlve  Action  quota. 
As  a  aeaber  of  the  Hiring  Coaalttee,  It  Is 
yaur  job  to  help  choose  a  successor  to  the 
post . 

The  only  woaan  who  has  appllad  for  the  job 
seeas  competent  and  aeets  the  written  qualif- 
ications for  the  job  but-  she  Is  olearly  less 
qualified   than  both  of  the  aen  she  Is  coapet- 
Ing  with.     Heabers  of  the  Coamlttee  disagree 
about  what  should  be  done:  soae  say  hiring  a 
woaan  Is  absolutely  necessary  for  breaking 
down  eaployaent  dlscrlalnatlon ;  others  say 
hiring  a  less  qualified  person  would  be  fool- 
ish as  well  as  unfair  to  those  working  under 
the  new  executive. 

To  have  a  full  hearing  of  all  views  on  this 
critical  Issue,  the  Hiring  Coaalttee  has 
asked  each  aeaber  to  prepare  a  carefully 
written  stateaent  to  be  distributed  In 
advance  of  a  aeetlng  to  discuss  the  Issue. 
Write  a  stateaent  which  represents  your  posi- 
tion m  the  aatter.  making  It  as  logical  and 
persuasive  as  possible.     Your  writing  task  Is 
to  persuade  the  Coaalttee  to  adopt  your  own 
view  and  to  vote  on  the  job  candidates  In 
aocordande  with  your  view. 


Mote  that  In  this  proapt.  the  underlying  topic  could  be 
described  as  the  Issue  of  afflraatlve  action  employment 
policies.     The  prompt  might  have  been  offered  In  this 
form  : 

PROMPT  31b 

Argue  pro  or  con  affirmative  action  policies 
In  admissions  or  employment. 


Instead,  the  task  was  se 
llf e-sltuatlon  In  which 
afflraatlve  action  were 
of  recommending  action. 


t  m  the  form  of  a  simulated 
the  writer's  attitudes  towird 
to  be  expressed  In  the  context 
Actually,  this  prompt  can  be 
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criticized  for  being  soaewhat  loadled:   The   phrase,  "is 
clearly  less  qualified   than  both  oT  the  nen  she   is  coni- 
peting  tfitht^   led  soae  students  to  assuae  that  the  tes- 
ter thought  the  reaale  applicant  should  not  be  hired. 
k  less  judgmental  wording  would  have  been:  "is  less 
experienced  than....** 

Sometiaes  even  pi lat- te st i ng  in  advance  fa  lis  to 
reveal   weaknesses  in  persuasive  writing  tasKS  and  these 
weakaesses  show  up  later  in  the  testing  of  large  popu- 
lation9*     Of  three  prompts  used  in  the   1970*s  on  a 
statewide  proficiency  test,*  only  one  net  the  important 
criteria  for  persuasive  prompts;  that  they  a)  be 
relevant  (i.e.  students  should  care  about  the  issue); 
b)  draw  on  an  adequate  information  base  (i.e.  students 
should  know  something  about  the  issue);  and  c)  be  non- 
biasing  (i.e.  different  positions  on  the  issue  nay  be 
defended  reasonably). 

On   this  statewide  proficiency  test,  over  three 
different  years,  students  were  asked   to  take  a  position 
on  these  issues:  a)   whether  physical  education  should 
be  a  required   course  ^^n  high  school  ,  b)  whether  firemen 
should  be  allowed   to  %o  on  strike,  and  c)   whether  stu- 
dents should  be  required   to   produce  a  writing  sample  as 
part  of  the  proficiency  test.     According  to  a  number  of 
teacher-dreaders,  the  first  issue  generated  wide  appeal 
and  students  generally  were  able  to  advance  valid  argu- 
ments on  both  sides  of  the  question.     On  the  second 
issue  teacher-readers  reported  that  fewer  students  had 
strong  opinions  or  even  understood  the  underlying 
issues  in  the  debate.     The   third  issue  proved  to  be 
relevant  but  somewhat  loaded;  that  is,  teacher^readers 
reported  that  no  arguments  against  the  writing  sample 
requirement  appeared  reasonable  to  them«     One  teacher- 
reader  believed  that  the  prompt  was  culturally  biased, 
claiming  that  students  who  argued  against  the  writing 
sample  requirement  came  from  particular  ethnic  groups 
that  did  not  share  mainstream  emphasis  on  the  impor- 
tance of  writing.     Being  unfamiliar  with  the  arguments 
for  using  a  writing  sample  to  measure  proficiency, 
these  students  were  unable   to  provide  convincing 
coun  ter  -arg  um  en  t  s  for  the  teacher-*readers,  all  of  whom 
were,   to  some  extent,  biased  in  favor  of  the  writing 
sample  requirement. 

In  short,  as  with  imaginary  situations  what  can 
become  "loaded  questions,"  problems  result  when  stu- 
dents are  aslied   to  take  a  pro  or  con  position  on  a 


(•CH3PE,   the  California  High  School   Proficiency  Exami- 
nation,    administered     by     the  Educational  Testing  Ser 
vice) 
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controversial   Issue  on  which  only  one  position  Is  con*- 
sldered  defensible  by  nost        all  of  the  teacher- 
readers.     Problerss  also  resalt  If  a  carefully  qualified 
position  that  is  neither   pro  nor  con  represents  a  aore 
intelligent  response.     In   these  cases,  students  oay 
feel   that  tne  proapt  does  not  allotf^then  to  straddle 
t^e  issue,  and  so  they  will  adopt  an  artificially 
simplistic  pro  or  con  view,  thus  failing  to  write 
either  convincingly  or  interestingly. 

Topics  With  Hidden  Biases 

In  attespting  to  guide  students  towards  acre 
fruitful   ideas,  a   test-maker  will  soaetiaes  load  a 
topic  with  words  that  unintentionally  suggest  the 
test-aaker's  biases.     Students  sensing  the  bias  aay 
then  write  froa  their  conception  of  what  the  test-aaker 
believes  rather  than  froa  their  own  perceptions  and 
opi  nions  . 

Tne   following  topic  was  pilot- tested  at  a  state 
university  as  part  of  an  equivalency  exaaination: 

PROMPT  32 

Consider  the   words  trees  and  plastic .  Write 
a  paragraph  in  which  you  discuss  some 
aspect(s)   of  what  the  words  suggest  to  you. 

Students  tended   to   fall   into  one  of  two  kinds  of  traps 
in  their  responses..     Soae  students  siaply  catalogued 
cliches  about  man's  abuse  of  the  environment,  using 
••trees'*   and  '•plastic"  to  represent  a  kind  of  polariza- 
tion of  the  natural  v^ersus  the  aan-aade.  Others, 
atteapting  to  go  beyond  cliches  and  generate  an  origi- 
nal  thesis,  siaply  got  bogged  down,  and  the  resultant 
writing  was  strained  and  even  incoherent. 

The   prompt  was  revised  to  read  siaply  trees .  In 
response  to  the  new  proapt,  students  produced  interest- 
ing essays  di splaying  an   enormous  range  of  reactions  to 
the  word  **tr«es'*,  a  range  that  Included  discussions  of 
family  trees,  branching  tree  diagraas,  wilderness 
treks  ,  the  uses  of  wood,   an^  man's  encroachment  on 
nature.     The  single  word  clearly  elicited  what  the  ori- 
ginal  topic  was  meant  to  inspire.     (Plastic  was  later 
used  alone  with  excellent  results  as  well.) 

Finally,  a  kind  of  unintentional  bias  occurs  when 
the  test-maker  assumes  that  students  share  certain 
attitudes  or  that  they  perceive  themselves  as  the 
test-maker  perceives  them.     Wnen  the  following  prompt 
was  piloted   far  a  college  proficiency  test  in   1980,  the 
test  committee  was  certain  that  the  prompt   would  have 
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universal   appeal   for  students.     Their  certainty  was 
bas9d  on  the  experiences  oT  their  own  late  adolescence 
during  the  turbulent,  deaonstration-niled   1960s  and 
703  .     They  assuaed  that  even  though  narches  and  pro- 
tests  now  sees  to  be  less  widespread,  all  young  people 
perceive  themselves  as  rebels  at  soae  time: 

PROMPT  33 

Even   those  of  us  who  oiay  conform  outwardly 
have  our  private,  personal   forms  of  reb el- 
lion.     Some  rebel   against  society's  conven- 
tions, soae  against  family  values,  and  others 
against  the  attitudes  of  peers.     tfe  might 
dress  in  unconventional  ways,  reject  our 
family's  traditions,  or  disagree  with 
friends'   political  beliefs.     Some  of  us  may 
simply  rebel  against  popular  fads.     Many  of 
us  are  not  interested   in  Jogging  five  miles  a 
day,   eating  natural  foods,  or  wearing 
designer  Jeans. 


I?ISTRUCTI0H3: 

Describe  one  way  you  choose  not  to  conform. 
Explain  why  you  act  as  you  do  and  analyze  the 
effect  your  choice  has  on  your  life. 


On  the  whole  students  had  little  to  say  in 
response  to  this  prompt.     Pew  had  actually  experienced 
"personal   forms  of  rebellion"  which  they  took  seriously 
and   thus  could  write  about  convincingly.     In  reviewing 
the  pilot  essays,  the  test  committee  felt  that  this 
group  of  students  was  less  concerned  with  rebelling 
than  with  conforming,  that  they  could  not  identify 
social  values,  mores,  or  fashions  which  they  wanted  to 
rebel  against   in  order  to  establish  their  own  identi- 
ties;  rather,   these  students  saw  theaselves  as  fitting 
in  to  the  present  life -styles  of  friends  and  fa allies. 
On   the  basis  of  the  pilot-testing,  the   test  committee 
chose  an  alternate  prompt,  one  which  asked   students  to 
write  about  making  a  difficult  decision.     The  response 
to   this  prompt  was  excellent;  it  had  tapped  a  core  of 
experience  imp:>rtant  to  this  particular  group  of 
s t ud en t-y r  i  ter  s  • 

to  avoid  biasing  effects  In  either  writers  or 
readers,   test**makers  need  not  abandon  thought-provoking 
questions  that  allow  students  to  explore  values  or 
investigate  ideas.     Mnen  evaluators  wish  to  test  the 
reasoning   skills  needed   for  exposition  or  argument. 
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tney  neej   not  avoid   using  challenging   state-nents  as 
pro.-npts  for   wr-itins.     Th-J   following   prompts  present^ 
generalizations  w^ich  can  be  supported  £r  challenged  in 
a  conarent  essay,   enabling   the   student  to  demonstrate 
Tiany  of  the   skills  needed   for  academic   writing,  while 
allowing   theij   to   draw  on   personal   experiences  and 
express   personal  values: 

PROMPT  3^ 

Man    Is  3   sreatjre  of  hablto 

Do  our  habits  nake   U3  aore  stable,  more  effi- 
cient,  or   Just  duller?     Discuss.  Support 
your   arguments  with   one  or   two  specific 

illustrations   frona   you^   reajlng   or  exp^rl- 


pao>ipr  3^^ 

3pposltlon   is   indispensable.     A  person  will 
often   learn  tore  froa  the   people  wno  disagree 
witn  him  or   her   than   fro-n   those   wno  agree. 

Do  you  agree  witn  this  observation?     Have  you 
profited  jore   fro-n   people  whose   views  w2re 
similar   to   yours  or   from   people  whone  views 
opposed   your   own?     Discuss,   using  one   or  two 
specific   examples  to   support   your  position. 


These   two  prompts  invite   an   exploration  of  ideas 
wiicn  need   n3t  be  argumentative  or   persuasive  in  tone 
or   purpose.     Further,   these  prompts  require   a  thought- 
ful  development  that  makes  them  appeal  more  to 
college-bound   or  older   students.     The   following  two 
p-ompt  s-'-nei  ther  dealing   with  cuTent  con  trover  91  es-- 
elicited   strongly  persuasive  writing   from   ninth  through 
twelfth  graders  at  all   ability  levels  in  a   Bay  Area 
nign  scn-^olL 

PROHPr  36 

Hot  all   inventions  have  been  good   for  human- 
ity,    Mame  one  invention  we  would  be  better 
off   without,   and  make   it  clear   why.     You  may 
do   trfis  writing  as   an  essay,   journal,  letter, 
story,   or  other  forta. 


PROMPT  37 


Imagine   that  a   saall  group  of  people  will  be 
sent  to   colonize  a  nei^  planet.     Food,  clotn- 
ing  ,   shelter   ani   transportation  have  been 
pro'/idei   for.     You  are  anong   those  asked  to 
select  a   few  additional  things  to  be  sent 
along   in   th?   limited   space  available   in  the 
ship.     Wnat  one  ite^i  would   you  reconsend,  and 
why?     You  may  write  your  recommendation  in 
the  fora  of  a   story,   a  dialogue,   a  letter,  a 
speech,   an   essay  or  other  form. 


Tne  responses  to  the   invention  prompt  ranged  froni 
serious  political   or  social   arguments  to  huuorous  pro- 
posals  sjggesting   abolition  of  such  dreaded  inventions 
as  the   waffle-iron  or   hair  curlers.     The  objects  recom- 
mended  for   the  colonizing  of  the  new  planet  included 
everything   from  important  books  intended   to  help  the 
new  community  avoid   further   war   to  a  Jar  of  peanut 
butter,   proposed   seriously  in  order   to  inspire  curios- 
ity and   com:Tient  among   the  new  colonists  and  thereby 
ensuT^e   a  certain   degree  of  levity  in   all  future 
decision-making. 
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THE  WRITING   INSTRUCTIONS:    PURPOSES  AND  PROBLEMS 


Section  Overview 

As  described  above,   the  writing  prompt  has  two 
elements — the   topic  which  states  the  required  content 
of  the  writing  (the  what)  and   the  Instructions  which 
give  the  suggestion  about  or  llaltatlons  on  the  content 
of  the  writing  (the  how)  .     This  second  tleaent  can  also 
be  called  alaply  **the  writing  Instructions."  The  writ- 
ing  Instructions  aay  be  very  simple,  consisting  merely 
of  the  phrase,  **Wrlte  about..."  or  they  may  be  quite 
elaborate  as  the  test-maker  tries  to  guide  the  writers 
in  their  responses.     As  with  cuing  for  content,  cuing 
for  form  can  help  assure  that  students  understand  what 
is  expected  of  them.     The  prompt-writer's  dilemma  is  to 
provide  enough  guidance  without  providing  too  much. 
This  final  section  considers  the  following  questions 
relating  to  auoh  instructions:  a)   Do  they  clarify  the 
test-maker's  intention,   leading  students  to  respond  in 
ways  the  scorers  will  reward?  b)  Do   they  effectively 
limit  what  the  writers  attempt,  making  writing  samples 
comparable?   c)  Are  the  task  restrictions  helpful,  or 
are  they  likely  to  distract  students  from  the  central 
task  of  communicating  their  ideas  in  writing? 

Hidden  Agendas  in  Instruction 

The  hidden  agenda  on  a  writing  test  is  the  set  of 
implicit  expectations  that  every  test<-maker  has  about 
how  the  student-writer  should  respond   to   the  prompt. 
Students  often  understand   these  ex  pec ta t ions ' b ut  some- 
times an  unintended   trap  is  created.     Pilot  testing  of 
prompts  will  usually  reveal  Cfses  in  which  the  test- 
maker's  assumptions  about  what  the  students  will  under- 
stand  prove  erroneous,  requiring  more  explicit  direc- 
tions.    Sometimes  what  is  needed  la  not  a  new  prompt 
but  simply  a  willingness  on  the  part  of  the  teacher^ 
readers  to  accept  a  broader  range  of  possible  responses 
than  were  originally  anticipated. 

There  are  three  common  traps,  however,  that  might 
be  avoided  by  a  careful   search  for  hidden  agendas: 

The  "Describe"  Trap 

An  interesting  photograph,  previously  used  suc- 
cessfully as  a  writing  stimulus,  was  presented  as  a 
test  promp'i  to  two  classes  accompanied  by  the  instruc- 
tions "Describe  what  you  see  in  the  picture  above."  One 
of  the  two  classes  had  been  accustomed  to  writing 
stories  and   essays  based  on  pictorial   stimuli.  They 
had  been  practicing  making  accurate  observations  and 
writing  descriptions  by  writing  simulations  of  police 


and   insurance  reports.     The  second  class  had  not 
engaged   In   slnilar  work.     The   students  in  the  class 
with  the  specialized   practice   in  description  wrote  the 
barest,  most  literal  descriptions  of  what  they  saw  in 
the  test  proapt  photograph,  refusing  to  aake  aany 
inferences  or  in ttr pr t ta t ions .     These  students  received 
low  scores  from  the  tester  because  their  writing  was 
the  least   interesting  and  imaginative.     The  tester's 
hidden  agenda  involved   a  definition  of  "descr iption" 
for  this  task  which  required  writers  to  be  imaginative, 
not  literal,  and  make  Inferences  about  what  might  be 
happening  and  about  the  mood  inspired  by  the  picture. 

The  '•describe'*  instruction  has  been  proble.natic  in 
other   assessments.     One  school   system   found   that  the 
task.   '*Descr ibe  a  person  who  is  important  to  you,"  eli- 
cited   from  many  writers  the  most  limited  physical 
descriptions,  while  a  revised   instruction,  "Tell  about 
a  person  who  is  important  to  you,**  seemed  to  suggest  to 
students  that  they  could  select  whatever  facts  about 
their  subject  interested  them  and  might  interest  a 
reader.     Some  teachers  argued   that  the  '•describe"  ver* 
sion  of  the  task  was  a  good   test  of  whether  students 
knew  what  it  meant  to  describe  and  that  description  of 
a  person  can  be  much  more  than   a  list  of  bare  physical 
characteristics.     But   in  this  case,  the   instruction  to 
"describe"   again  made  the  test  more  a  measure  of  the 
students*   ability  to  recognize   the  test-maker's  meaning 
rather   than  a  measure  of  their  own  ability  to  write 
Interestingly  about  a  person  important  to  them. 

In  general,   it  can  beaaid  that  any  writing 
instruction  consisting  only  of  the  simple  direction, 
'•describe,'*   is  a  ease  of  underprompting  and  creates  an 
unfair  hidden  agenda  unless  cues  are  added  to  suggest 
what  the  tester  ciesires  to  see.     Prompts  such  as, 
"Describe  your  favorite  person,"  "Describe  your  favor- 
ite object,"  ••Describe  your  favorite  food,"   are  popular 
topics  with  students,   and  can  be  made  more  indicative 
of  the   tester's  expectations  by  adding  a  qj^l iflcation 
such  as  the  one  Illustrated  by  the  underlining: 

PROMPT  38 

••Describe  your  favorite  (person, 
food,  object)   so   that  your  reader  will  under- 
stand  why  it  is  your  f  avor  i  te  .•• 


This   simple  addition  clarifies  the  rhetorical  purpose 
for  the  writers  and  cues  them  to  recognize   that  the 
test-maker   is  interested   in  more  than  merely  the  abil- 
ity to  write  accurate  physical  descriptions. 
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Other  casualties  of  und er prompt  1 n g  from  using  only 
the  unaodlfied   instruction,  "describe",  occurred  In  two 
tests  asking  students  to  describe  places:  "Describe 
your  home"  and  "Describe  your  school."     The  problem 
with  these  prompts  Is  that  they  are  devoid  of  rhetori- 
cal  purpose.     Thus,   the  descriptive  task  becomes 
overwhelming  without  the  limiting  Instruction  to  choose 
a  favorite   something  and  explain  one's  preference  to 
someone.     The  atudents.   In  effect,  must  set  their  own 
limits  and  construct  their  own  rhetorical  purposes  as 
follows:  Wny  am  I  describing  ay  house  or  my  school? 
Who  wants  to  know  what?     How  can   I  possibly  describe 
all  of  lt--and.  If  not  all,  then  what  aspects  should  I 
a  tteapt? 

Some  students,   familiar  with  such  vague   tasks,  do 
quickly  impose  their  own  rhetorical  purposes  and 
describe  homes  or  schools,  evoking  feelings  about  these 
places.     Most  students,  however,  painfully  and  con- 
scientiously begin  a  -guided  tour  of  their  home*  or  try 
to  portray  graphically  the  outside  of  the  building, 
giving  facts  about  its  location,  its  height,   its  color, 
etc.     In  this  case,  providing  the  additional  instruc- 
tion: "Describe .. .so  that  your  reader  can  understand 
what  it  looks  like"   isn't  very  helpful.     The  writers 
are  still   restricted  to  providing  a  physical  descrip- 
tion of  a  large,  complex   structure  that  they  are  not 
looking  at.     They  would   still   not  be  sure  whether  their 
exanlners  expected  them  to  convey  feelings  and  atti- 
tudes or  "just  the  facts." 

It   is  Interesting  thai;  one  major  research  study 
foundered   when  an  entire  group  of  subjects  In  eleventh 
grade  simply  refused  to  write  on  the  following  prompt 
because  they  felt  it  was  too  "childish"  (Stahl,  1977). 

PROMPT  39 

My  Home:     Write  a  deicription  of  your  home, 
it!  rooms  and   their  contents  in   such  a  way 
that  someone  who  has  not  visited  it  can  form 
an   idea  of  the  way  it  looks. 

What  the  researcher  did  not  realize  Is  that  it  was  also 
too  difficult,  that  only  the  most  mature  and  confident 
writers  can  take  such  an  an  undeveloped  subject  and 
provide  their  own  point  of  view,  their  own  definition 
of  the  task,  their  own  rhetorical  purpose  and  thereby 
create  a  coherent  and  meaningful  piece  of  writing. 

A  sophisticated  version  of  the  "describe  an 
object"   prompt  with  detailed   instructions  has  elicited 
good  expository  writing   in  state  college  writing 
assessments  conducted  for  the  purpose  of  gathering 
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information  for  making  adaissiona  deciaions  (Irfnite. 

1973): 

PROMPT  40 

Many  observtrs  of  our  society  claim  that 
modern  man,   immersed  in  material  isra »  ia 
"owned  by  his  objects."     Yet  many  of  us  have 
objects  that  we  treasure  not  just  for  their 
material  value  but   for  a  variety  of  other 
reasons.     Assignment;     Describe  one  pr  more 
objects  which     art  Important  to  you»!  Explain 
what  values  they  represent  and  comment  on 
those  values. 


Walker  Gibson  {1959),  m  Seeing  and  Writing,  pro- 
vides a  class  assignment  to  describe  a  church: 

PROMPT  «1 

Find  a  church  in  your  neighborhood  that  seems 
to  you  an   interesting  building.  Describe 
what  you  see  on  its  facade  in  detail  and  in 
such  a  style  that  your  reader  sees  it  as 
interesting  too . 


He  goes  on  to  explain  what  a  complex  task  this 
apparently  simple  description  is,  and  why  a  church  is  a 
good  subject  for  this  kind  of  practice.     He  suggests 
that  churches  are  often  lo'aded   with  personal  meanings, 
and  that  the  task  entails  working  out  "who  you  are,  who 
you  should  be,  as  you  speak  to  a  reader  about  the 
appearance  of  a  church*"  Description  must  be  done  with 
both  a  pjrpose  in  mind  and  a  point  of  view  in  order  to 
re-create  the  actual  experience  of  looking.  Descrip- 
tion is  a  sophisticated  exercise  even  for  advanced 
writers;  it  is  not,  as  might  be  expected,  the  simplest 
mode  of  writing  and  therefore  It  must  be  used  judi- 
ciously in  testing  for  general   fluency.  (Gibson's 
church  description  assignment  appears  as  Theme  8  in  his 
course   sequence  of   15  exercises.) 

Even  when  the  goal  of  assessment  is  primarily  to 
test  students'   ability  to  develop  an  objective  descrip- 
tion rather  than  to  express  personal  meanings  through 
descrtpuion,  the   instruction,   "describe",  can   still  be 
a  oase  of  und er prompting  : 

PROMPT  M2a 

Describe  wnat  you  see.  (Accompanying  a  photo 
and  diagram  of  an  automobile  accident.) 


-IS? 


169 


Soae  students  tfill  write  about  the  horror  of  the 
accident,  others  will  try  to   Infer  who  Is  to  blarae  or 
will  write  about  what  must  hive  happened,  while  a  few 
will  try  to  give  a  precise  description  of  the  position 
of  cars  and  bodies.     If  test-aakers  wish  to  elicit  ,this 
last  kind  of  writing,  thtir  instructions  must  be  more 
explicit,  specifying  a  rhetorical  purpose: 


PROMPT  U2b 


Study  the  accident  in  the  photo,  using  the 
diagra'B  to  help  you  gain  a  clear  understand- 
ing  of  the  situation.     Describe  the  accident, 
reporting  details  accurately  so  that  a  Jury 
or  insurance  coapany  can  aake  a  fair  decision 
about  who  is  to  blaae. 


Tnis  task  is  a  powerful   test  of  aany  skills  that  con- 
tribute to  writing  ability.  Including  the  ability  to 
aarshal  facts  and  observe  details  in  a  reasonable  way 
so  that  a  reader  can  grasp  the  Important  aspects  of  a 
situation — a  valuable  skill  worth  testing. 

Such  tests  can  be  scored  hollstically  as  well  as 
analytically,  of  course.     But  student  writers  must 
share  a  common   understanding  of  the  rhetorical  problem 
they  are  trying   to   solve  if  teacher-readers  are  to  make 
fair  comparisons  of  their  solutions. 

The  Special  Conventions  Trap 

It   l5,  of  course,  not  a   trap  to  test  students  for 
their  knowledge  of  special  conventions,  such  as  the 
forms  of  business  letters  or  friendly  letters,  as  long 
as  the  test-maker  does  so  deliberately*  deciding  In 
advance  which  forms  are  acceptable  and  how  deviations 
are  to  be  scored.     Teachers  and  students  must  be 
Informed  of  the  conventions  which  will  be  tested   In  the 
writing  assessment.     The  trap  Is  present  only  when  cer- 
tain conventions  are  sprung  upon  unwary  teachers  and 
St  uden  t s • 

In  an  effort  to  create  a  meaningful  rhetorical 
context,  test-makers  may  call   for  students  to  write 
business  letters,  but  fall  to  decide  In  advance  how  to 
weight  formal  considerations  such  as  salutations*  clos- 
ings, return  addresses,  etc.     Such  a  problem  aroa^e  with 
this  otherwise  excellent  writing  prompt. 

PROMPT  M3 

The   following  advertisement  appeared   In  the 
local  paper: 
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SUMMER  EMPLOYrtENT 

Earn  Si  Learn 

On-the-job  training  for  future  positions  in: 

Medical  Services  Aida  at  Hijhland  Hospital 

Ticket   taker  at  local  theatre 

Grocery  clerk  at  local  supermarket 

PI  ufabing  Assistant  for  plumbing  company 

Assistant  to   the  manager  of  a  circus 

Earn  $3.50  or  more  per  hour  while  you  learn  a 
valuable  akill  . 


Send   letter  of  application  to: 

Hs.   Laura  Jones,   Opportunity  Training  Center 

2212  Fruitvale  Ave.,   Oakland,  California 

9^231 

Pretand  that  you  are  applying  for  one  of  the 
Jobs  listed.     Write  a  latter  explaining  why 
you  should  be  hired.     Tou  should  give  infor- 
mation about  yourself.     The  information  may 
be  actual   facts,  made-up  information,  or  a 
combination  of  fact  and   fiction.     The  letter 
should  help  you  get  the  job. 

Students  wrote  lively  and   interesting  prose,  using 
real  or  imaginary  facts.     But  they  also  used  a  variety 
of  different  conventions  i^n  setting  up  their  letters. 
Teacher  readers  differed  In  their  preferred  conven- 
tions; some  believed  that  '•correct  form**  called   for  no 
paragraph  indentation.     Still  other*  stressed  the 
importance  of  adequate  spacing  between  the  date  and  the 
body  of  the  letter. 

If  all  students  have  taken  a  course  in  business 
English  or  typing  and  have  been  taught  the  same  set  of 
conventions  for  business  letters,  it  would  be  possible 
to  devise  some  system   for  scoring  the  finer  points  of 
layout  and  presentation.     However,  it  is  clear  that 
holistic   acoring  would  not  be  the  most  efficient  or 
accurate  way  of  measuring  mastery  of  these  conventions. 
A  rating  of  letter  form  might  be  an  appropriate  addi- 
tion after  the  holistic  assessment  has  been  made. 

In   the  assessment  for  which  this  prompt  was 
designed,   testers  were  not  concerned  with  conventions 
or  consistent  advance  instruction  to  students  in  the 
fine  points  of  the  business  letter.     Rather,  their  pur- 
pose was  to  determine  whether  students  could  express 
themselves  adequately  in  a  life-like  situation. 
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Assuming  that  students  aight  refer  to  one  of  several 
business  letter  guides  should  they  actually  wish  to 
write  a  letter  applying  for  a  Job,  the  teachers' 
greater  concern  was  whether  students  could  Include 
appropriate  Inforaatlon  about  themselves  and  write  In  a 
convincing,  personable  style  that  would  apptal  to  pros- 
pective employers.     Obviously  students  needed  some  con- 
trol of  punctuation  and  spelling,  of  vocabulary  and 
sentence  structure.  If  they  were  to  sound  mature  and 
reasonably  well-educated.     But   In  the  holistic,  rapld- 
Impresslon  scoring,  these  factors  could  easily  and 
fairly  be  weighted  for  what  they  contributed  to  the 
over-all  effectiveness  of  the  letter.     Since  no  one  was 
really  applying  for  a  Job  as  a  business  secretary,  the 
use  of  particular  conventions  would,  even   In  real  life, 
be  less  Important  than  the  overall  impression  conveyed 
by  the  writer.     In  these  circumstances,  teacher-readers 
agreed  to   suspend  debate  on  formal   conventions  and  rule 
out  of  order  any  penalties  Imposed   for  Incorrect  letter 
etiquette . 

tfnen  offering  such  topics,  testers  should  note 
that  m  the  absence  of  specific  directions  to  attend  to 
conventions,  students  may  differ  In   their  assumptions 
about  what  the  tester  wants.     Students  understand  that 
this  is  an   Imaginary  letter,  not  a  real  one.     Some  of 
the  best  students  may.   under  the  circumstances,  drop 
all  but  the  barest  suggestion  of  a  letter:     "Dear  Sir." 
and  "Sincerely."   being  the  extent  of  their  formalities. 
Other   students  may  pay  close  attention  to  letter  forms 
assuming  that  this  Is  what  Is  being  tested,  and  may 
offer  only  a  alnlmum  amount  of  content,  as  In  this 
example  of  Impeccable  form: 
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WRITING  SAMPLE: 


1204  Kenyon  Road 

Oakland.  California  94724 

March  2.  1978 


Ms.  Laura  Jones,  Director 
Opportunity  Training  Center 
2212  Fruitvale  Avenue 
Oakland,  California  94321 

Dear  Hs .  Jones: 

I  aa  very  interested  in  applying  for  the 
position  of  Medical  Services  Aide  at  Highland 
Hospital,  which  was  advertised  In  the  OAKLAND 
TRIBUTE  last  Saturday. 

I  as  sixteen  years  old.  and  have  had 
experience  baby  sitting,  attending  old  people, 
and  volunteering  as  a  Candy  Striper, 
I  have  excellent  references,  which  I  have 
enclosed  on  a  separate  sheet.     I  an  available 
for  an  interview  any  day  after  2:00.     My  phone 
number  is  634-0972. 

Sincerely  yours, 
Pat  Saith 


No  one  could  find   fault  with  the  formal   aspects  of 
this  letter  of  application;  however  some  teacher- 
readers  felt  it  suffered  by  comparison  with  the  letters 
of  students  determined   to  win  the  Job  in  this  one  com- 
munication act  without  recourse  to  references  or  inter- 
views.    Some  of  the  fuller,  more  personal  of  these 
letters  were  strong,  impressive  pieces  of  writing, 
although  strictly  speaking  they  revealed  less  awareness 
of  appropriate  business  formalities  than  the  letter 
above. 

Teachers  may  Justify  teaching  conventional  organi- 
zational formulas,  such  as  the  five-paragraph  essay, 
believing  that  they  are  providing  students  with  struc- 
tures to  order  their  ideas  for  greater  readability. 
But  the  evaluators  are  not  Justified  in  penalizing  stu- 
dents for  not  using  those  forms  in  an  assessment  of 
composing  abilities.     Instead,  the  major  concern  is 
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wnether  stuients  can  achieve  their  rhetopieal  purpos-s 
by  whatever  aeans  thty  choose.     Holistic  assessment 
puts  reader  and  writer  Into  a  naturalistic  relation- 
ship, one  In  which  the  reader  is  allowed   to  respond  not 
as  an  error-counter  or  a  con»erver  of  threatened  forms, 
but  as  a  receiver  of  Intended   comaun lea tlon . 

The  Plek-a-Fom  Trap 

Soaetlmes     a  proapt  will   atteaspt  to  encourage  stu- 
dents to  txperlaent  with  fictional   foras  or  to  allow 
thsa  aaxlaua  freedoa  to  aatch  fora  to  Intention: 

INSTRUCTIONS: 

Write   In  whatever   fora  you  want — journal. 
lett«r.   essay,  story. 


This  kind  of  Instruction  with  a  range  of  options  gives 
opportunity  for  the  consideration  of  techniques  which 
may  be  In  the  process  of  being  taught  In  the  writing 
program.     Thus.   Imaginative  students  are  allowed  a  full 
range  of  choices  for  creative  experimentation  which 
often  results  In  outstanding  writing.     Further,   the  use 
of  such  a  test  Instruction  encourages  teachers  to  pro- 
vide opportunities  In  class  for  students  to  practice  a 
variety  of  forms. 

But  the  apparent  freedoa  of  this  Instruction  may 
becoae  a   trap  If  the  readers  who  score  thes?  papers 
have  a  "hidden  agenda"   that  rewards  conventional 
school-essay  language.     Students  who  choose   to  write 
letters  or  journals  aay  laltate  the  speech  of  their 
characters  and  write  less  formally  or  correctly  than 
they  would  have  If  thty  had  chosen  more  conventional 
essay  or  story  forms.     The  tester  must  be  prepared  to 
train  teacher-readers  to  score  experimental   papers  so 
that  the  student  Is  not  penalized  for  attempting  the 
very  fictional  techniques  that  have  been  suggested  In 
the  writing  Instructions. 

A  wide  choice  of  genres  can  have  another  unfor- 
tunate effect.     Even  when  ttacher-resders  art  conscien- 
tious about  giving  full  credit  to  foras  other  than 
traditional  essays,  thty  aay  find  It  difficult  to  com- 
pare papers  of  different  genres.     It  Is  easier  to  judge 
the  relative  merits  of  two  short  stories  than  It  Is  to 
judge  the  relative  merits  of  a  short  story  compared 
with  an   argument.     Hany  experts  In  writing  assessment 
warn  against  providing  a  wide  range  of  foras  because  It 
is  likely  to  decrease  Inter-rater   agreement.     When  the 
purpose  of  the  assessment  Is  to  encourage  the  teaching 
of  writing  for  different  purposes,  or  writing  with 
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different  forms,  a Jnlnlstrators  of  assftssaents  should 
consider  offering  several  writing  tasks,  each  clearly 
calling  for  a  different  mode  of  writing. 

In  practice,  however,  the  given  topic  soaewhat 
Halts  the  genre  and  rhetorical  strategy  choices,  so 
that  papers  In  a  particular  year  can  be  scored  quite 
reliably,  even^though  responses  range  across  a  variety 
of  foras.     In  the  case  described  above  (In  "Hypotheti- 
cal Situations  As  Topics"),  the  option  to  select  foras 
actually  helped  students  In  certain  years  to  solve  spe- 
cial probleas  Inherent  In  that  year's  topic. 

One  further  warning  relates  to  poetry  as  a  fora  of 
response.     Many  schools  do  not  have  large  nuabers  of 
teachers  who  could  confidently  evaluate  and  score  poe- 
try; even  poetry  teachers  find  It  hard  to  rank  order 
poeas  In  comparison  with  prose  works.  Therefore, 
schools  find  It  best  for  purposes  of  general  assessaent 
to  exclude  poetry  as  a  possible  aode  of  response. 

Helpful  and  Unhelpful  Instructions ; 

Suggestions  That  Becoae  Constraints .     Soaetlaes  In 
an  attempt  to  avoid  the  hidden  agenda  problem,  test- 
makers  strive  to  make  explicit  every  criterion  the 
writer  should  know  about,  to  provide  every  conceivable 
cue  that  alght  elicit  a  good  response.     If  teacher- 
readers  are  expected  to  reward  abundant  use  of  details 
and  personal  feelings,  the  test-maker  might  write: 

INSTRUCTIONS: 

Be  sure  to  Include  lots  of  details  and  to 
aake  your  feelings  clear. 


For  the  writer,  however,  what  Is  Intended  as  a  sugges- 
tlon  Is  transforaed  Inescapably  Into     ■  constraint 
which  aay  or  aay  not  be  helpful.     An  Instruction  that 
Is  Intended  to  provoke  more  richly  textured  writing, 
"Use  lots  of  details,"  may  Imply  tha..t  the  students  who 
do  not  will  be  penalized.     The  In str vie tlons  "Support 
your  opln'lon  with  specific  exaaples,"  or  "Write  a 
well-organized  essay,"  assuae  a  certain  knowledge  of 
structure  on  the  students'   part. and  serve  as  reminders 
to  them  to  use  than  knowledge  as  they  coapose.  Sush 
Instructions  are  expected  to  direct  their  attention  to 
what  It  means  to  write  ■  well-organized  essay,  to  con- 
sider the  usR  of  Illustrative  anecdotes  rather  than 
unsupported  arguments  or  lll-foraed  generalizations. 
For  aany  students  these  cues,  coming  In  the  context  of 
a  aehool  setting  In  which  they  have  practiced  respond- 
ing to  similar  Instructions,  can  be  helpful.  Since 
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they  trigger  aeaories  of  other  school  tasks  which  have 
been  evaluated,  such  instructions  aay  provide  students 
with  soae  idea  of  how  this  particular  task  will  be 
Judged  . 

But  two   things  can  turn  a  helpful   cue   into  a 
damaging  constraint.   Students'   concern  with  doing  as 
the  tester  asks  aay  keep  thea  from  attending  to  their 
own  construction  of  aeaning  in  response   to  the  topic, 
and  this  can  cause  them  to  produce  strained,  awkward 
essays  rather  than  the  fluent  pieces  of  which  they  may 
be  capable.     Or  students  who  attend   to   the  spirit 
rather   than  the  letter  of  the  topic  may  produce  out- 
standing responses,  but  be  scored  down  because  the 
scoring  guide  specifies  that  top  scores  should  be 
awarded  only  to  students  who  solve  the  problem  in  the 
way  specified.     The  following  constraints  are  likely  to 
divert  the  student  writers'   attention  from  communica- 
tion to  secondary  matters  and  so  prevent  them  from 
accomplishing  the  primary  writing  task. 

Effects  of  Overproapting.     Attempting  to  make  up 
for  the  deficiency  of  a  test  prompt  such  as  "Describe 
your  school"   or  "Tell  what  you  see  in  the  picture," 
■any  tesc-aakers  add  a  stream  of  instructions  and 
suggestions  which  sometimes  have  counter-productive 
effects  on  the  writer.     Three  examples  demonstrate 
these  elaboration  techniques  which  tend  not  only  to 
turn   the  writing  test  into  a  reading  test,  but  to  bur- 
den the  writer  with  choices  which  may  prevent  simple 
and  natural  responses. 

One  tester  sought  to  avoid   the  un d er prompt ing 
problem  of  "Describe  your  school,"  by  listing  aspects 
of  the  school  which  might  be  described. 

PROMPT  ««t 

Describe  your  school.     Tell  how  it  looks  from 
the  outside.     Tell  how  your  classroom  looks. 
You  may  descriZje  what  happens  in  the  hallways 
at  different  times  of  day,  or  you  aay 
describe  the  lunchroom  or  playground.     If  you 
like,  you  may  describe  the  people  in  your 
sc hool  . 

This  type  of  o v er prompt i ng  appears  when  testers  seek  to 
insure   that  writers  supply  adequate  details.  However, 
this  elaborated  cuing  encourages  an  unfortunate  ten- 
dency in  many  young  writers  to  describe  superficially 
many  different  things  rather  than  tc  concentrate  on  one 
aspect  of  a   subject  long  enough  to  write  effectively. 
The  additional   cues  might  not  be  so  destructive  of 
coherence  and  purpose  if  they  were  reworded  to 
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encourage  students  to  focus  on  one  part  of  the  school 
(See  the  discussion  under  ••Focusing."  below).  As 
Prompt   4M  Is  worded,  students  who  begin  conscientiously 
to  *tell  how  It  looks  from  the  outside'^  may  rarely  get 
any  further  than  that.     They  night  write  much  more 
interestingly  if  they  had  chosen  to  begin  to  describe 
the  playground  or  the  lunchroom.  The  student  might  also 
be  more  helpfully  guided   if  the  instructions  were 
rewritten  to  invite  them  to  describe  one  part  of  the 
school   in  a  way  that  would  be  helpful  to  a  new  student 
who  is  about  to  enroll  there. 

A  second  kind  of  over  prompt ing  can  result  when 
testers  wish  to  .insure  that  they  will  get  a  particular 
kind  of  writing,  knowing  that  students  may  be  misled  by 
such  a  prompt  as  '•Describe  what  you  see  in  the  pic- 
ture."    The  National  Assessment  of  Educational  Progress 
moved   away  from  holistic  assessment  toward  primary 
trait  scoring  precisely  because  the  test-maktrs  could 
not  predict  that  all  students  would,  for  example,  nar- 
rative or  to  write  imaginatively  in  response  to  a  pic- 
torial stimulus.   Invariably,   such  stimuli  would  Inspire 
some  students  to  write  discursively  or  factually. 

In  their   1973-7M  assessment,  the  NAEP  offered 
writers  a  photograph  of  children  playing  on  a  boat. 
The  Instructions  read: 

PROMPT  M5 

Look  carefully  at  the  picture. 

These  kids  are  having  Vun  Jumping  on  the 
overturned  boat.     Imagine  you  art  one  of  the 
children  In  the  picture.     Or   If  you  wish. 
Imagine  that  you  are  someone  standing  nearby 
watching  the  children.     Tell  what  Is  going  on 
as  he  or  she  would  tell  It.     Write  as  If  you 
were  telling  this  to  a  good  friend.  In  a  way 
that  expresses  strong  feelings.     Help  your 
friend   feel  the  experience  too.     Space  Is 
provided  on  the  next  three  pages. 


When  the  researcher  tested  this  prompt  on  a  group 
of  teachers,  one  of  them  complained  that  It  sounded 
like  the  old  Jibe,   "Keep  your  tar  to  the  ground,  your 
nose  to  the  wheel,  and  your  shoulder  to  the  grindstone: 
now  try  to  work  In  that  position."     Perhaps  the  prompt 
can  be  Justified  for  NAEP*s  purposes,  since  they  are 
testing  students  from  enormously  varied  backgrounds  for 
a  particular  ability  (i.e.  the  ability  to  tnter  Into 
the  Imaginary  world  of  the  picture).     But  this  prompt 
makes  m^ny  demands  on  the  writer  which  may  be  at  odds 
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with  the  writer's  spontaneous   response   to   the  picture. 
Simple,  relatively  open   instructions,   like  those  accom- 
panying a  picture  stimulus  In  the  Jefferson  County, 
Colorado,  writing  assessments  (1976  &  1977)   should  be 
considered  : 

PROMPT  46a 

Look  at   the   picture   above.     You  can  react  to 
It  In  any  way  you  wish  by  writing  about  It  In 
one  of  the  following  ways:     as  a  story  (Wnat 
have  the  characters  been  doing?  What  dould 
happsri  next?),  as  a  description,  or   as  an 
opinion. 

If  testers  wished   to  rule  out  the  possibility  of  mere 
factual  description,  which  is  likely  to  produce  writing 
that  Is  less  Interesting  than   stories  or  essays,  the 
Instructions  could  read  instead: 

,  '  PROMPT  461) 

Look  at  the  picture  above.     Let  it  give  you 
ideas  or  create  a  mood.     Then  use  the  Ideas 
or  the  mood  to  write  a  story,  a  memory  of 
something  that  happened   to  you,  or  an  essay 
or  argument  based   on  some  strong  feeling 
Inspired  by  the  picture. 

Since  this  kind  of  Instruction  opens  up  a  large  number 
of  choices  for  writers,   they  must  be  provided  with  ade- 
quate time  to  choose  their  point  of  view  and  mode  of 
r espon  se  •  • 

A  third   type  of  over  prompt Ing  can  occur  when  tes- 
ters wish  simply  to  remind  students  about  aspects  of 
their  training  which  should  be  applied   to   the  current 
task.     The   following  prompt   used   in  a  district  assess- 
ment illustrates  this  kind  of  over  prompting : 
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•Trie  NAEP  prompt  atteapts  to  provide  a  single  mode  and 
point  of  view  partly  to  help  the.  writer  begin  to  write 
more  quickly,  but  primarily  to  limit  the  range  of  like- 
ly responses.  These  goals  in  themselves  may  be  suspect 
if  the  writing  test  is  an  attempt  to  measure  how  stu- 
dents actually  compose  when  faced  with  an  opportunity 
to  writa  about  an   interesting  stimulus. 
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PROMPT  ^7 


"No  oian  is  an  island,  entire  of  himself. 
Every  man   is  a   piece  of  the   Continent,   a  part 
of  th2  nain..." 

John  Donne 


This  topic  should  lead  you  to  think  about  the 
possible  meaning  of  the  quotation.     You  may 
build  upon  the   idea  by  supporting  it.  or  you 
may  contra diet  the  topic  with  an  opposing 
point  of  view.      Attempt   to  build  a  logical, 
reasonable  opinion  paper.      Include  an  intro- 
ductory and  concluding  paragraph     and   two  or 
three  paragraphs  in   the  body  of  the  papar. 
The   supporting   paragraphs  aay  be  developed  by 
comparison/ contra St.  definition,  illustra- 
tion, or  any  oth^r   form  which  will  produce  a 
unified  essay  on  this  topic. 


This  prompt  was  pilot  tested  and  used  successfully  by 
conventional  measurement  standards:   it  elicited  papers 
that  holistic   scorers  were  able  to  distribute  along  a 
full   range  of  points  from  best  to  worst.     Bjt  the 
extent  to  which  students  benefited   from  these  elaborate 
instructions  is  unclear.     Current  research  on  topic 
effects   promises  to   provide  insight   into  what  Informa- 
tion students  use  when  they  approach  assessment 
pr ompt  s  .  • 

Specifying  Number  of  Words  ,  Sentences .  Paragraphs . 

In  most  writing  assessments,   test-makers  need  to 
Impose  a   time  constraint  which  Indirectly  controls  the 
length  of  student  work.     Some  assessors,   in  addition, 
impose  a  length  constraint  in   the  fora  of  a  minimum 
number  of  words,  sentences,  or   paragraphs.     It  is  not 
uncommon  to  see  : 

INSTRUCTIONS: 


Write  at  lease  three  sentences... 

Write  one  paragraph  describing... 

Write  two  paragraphs  in  which  you. 

Write  an  essay  of  about   300  ^ords 

0 


( el emen  tar  y ) 
( elementary) 
.(junior  high) 
.{high  school , 
col  lege) 


*See  Chapters  2,  3,  A  and  5. 
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In  general  such  constraints  are  unhelpful,  both  to  stu- 
dents and  testers.   The  testers'   purposes  can  be  served 
in  other  ways  that  do  not  interfere  with  the  priaary 
aia  of  the  testing,  which  is  to  observe  how  auch  and 
how  well  a  student  can  write  in  response  to  a  proapt. 

T;.e  instruction  in  the  elementary  school  to  write 
three  sentences  is  intended  to  insure  that  the  tester 
can  determine  the  extent  to  which  children  understand 
what  a   sentence  is  and  how  sentences  are  marked.  This 
aim  is  aore  appropriate  to  objective  testing  in  which 
testers  can  measure  sentence  recognition,  punctuation 
skills,  etc.,  as  skills  separable  froa  the  act  of  com- 
posing.    But,  in  an  actual   trial,  because  the  prompt 
asked   for  three  sentences,  several  third  grade  students 
whose   sentence  combining  skills  outstripped  their  peers 
were  penalized   for  writing  one,  long,  layered  sentence 
which  included  more  information  than  appeared  in  the 
three-sentence  papers.     Other  students  could  not  think 
of  three  sentences  on  the  given  topic   ("D^rscrlbe  a  ham- 
burger"), and  so  they  added  a  sentence  about  lome  other 
kind  of  food  (such  as  "chocolate  cake"  or  "hot  dogs") 
and  were  penalized   for  being  off  topic.     One  student 
even  construed  the  instruction  to  mean  that  she  should 
copy  her  one  sentence  over   three  tiaes.     Most  of  the 
third  graders  In  the  study     siaply  Ignored  the  three- 
sentence  constraint  and  wrote  one  to  six   sentences  of 
varying  quality  about  haaburgers.     The  aajority  of  stu- 
dents wrote  two  sentences. 

A  aore  likely  way  to   insure  that  students  will 
generate  aore  than  one  sentence  is  to  give  thea  an 
interesting  rhetorical  problem — one  which  provides  soae 
context  to  the  request  for  a  description  of  a  ham- 
burger.    In  any  case,  one  of  the  most  important  things 
a  tester  needs  to  measure  in  elementary  school  writing 
is  the  degree  of  flu§ncy--the  number  of  sentences  or 
ideas  the  child  can  generate  when  given  a  stimulating 
topic  to  write  on.     For  children  who  can  only  think  of 
one  sentence  to  write,  the  command  to  write  three  sen- 
tences is  simply  an  assurance  of  failure  before  they 

begin  if  they  even  notice  the  instruction.     These  low 

fluency  students  will  be  identified  whether  or  not  the 
constraint  on  length  is  included   In   the  test  instruc- 
tions.    In  a  test  the  instruction  to  write  a  set  number 
of  sentences  on  a  topic  not  only  may  encourage  redun- 
dancy,  it  can  increase  writing  anxiety  and  turn  stu- 
dents'  attention  to  arbitrary  formulas  for  composing 
rather   than  help  them  generate  ideas. 

The  problem  in  specifying  "Write  one  paragraph"  is 
somewhat  different.     If  the  focus  of  teaching  has  been 
the  construction  of  paragraphs,   and   if  students  have 
been  practicing  this  unit  of  discourse,  a  tester  may 
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well  justify  a  test  which  gives  thea  a  faaillar 
instruction  and  tests  for  paragraph  recognition  and 
constr U3 tlon .     But   the  virtues  of  this  approach  both  to 
teaching  and   to  testing  are  debatable*     Sooe  students 
will  already  be  sufficiently  fluent  to  produce  material 
for  two  or  more  paragraphs  even  before  a  teacher  intro- 
duces the  class  to  the  concept  of  paragraph  construc- 
tion.    Others  will   still  be  struggling  to  generate 
enough  taattrial   to  elaborate   a  single  idea.  Teaching 
the  paragraph  as  a  complete  unit  of  discourse  rather 
than  as  a  aethod  of  shaping  the  flow  of  ongoing 
discourse  is  alsleadlng.  even  at  early  stages  of  writ- 
ing  Instruction.     More  fluent  students  can  be  taught  to 
indent,  breaking  up  longer  pieces,   as  an  aid   to  reada- 
blllty.     Less  fluent  students  can  be  taught  to  ela- 
borate,  and   can  be  helped   to   understand   soae  of  the 
questions  that  readers  wanting  to  know  more  about  their 
subject  Bight  ask. 

On   any  test  of  general   fluency,   the  crucial  vari- 
ables will  be  whether  students  are  able  to  write 
fluent,   coherent  discourse  that  communicates  their 
intentions.     Performance  will  depend  on  response  to  the 
topic,   and  paragraphing  will   depend  on  the  extent  to 
which  they  have  learned   to  use  the  paragraph  to  struc- 
ture their   thinking  or  aid  their  readers.     The  instruc- 
tion ••Write  one  pa  r  agr  a  ph  .  .  .     or  "Write  a  paragraph..." 
introduces  a  concern  for  the  paragraph  as  a  unit  of 
testing  rather  than  as  a  unit  of  comaun Icatlon .  It 
leads  a   student  to  think:  "Wnat  kind  of  paragraph  does 
ay  teacher  want?     Suppose  what  I  want  to  say  doesn't 
fit  into  one  paragraph?     Snould  I  change  ay  idea,  go 
ahead  and  write  two  paragraphs,  or  just  skip  the  inden- 
tation?" 

Again." as  with  specifying  a  certain  number  of  sen- 
tences,  if  testers*   intentions  Include  aeasurlng  the 
ability  to  recognize  where  to  indent  or  the  student's 
sense  of  the  internal  coherence  of  a  good  paragraph, 
they  can  develop  reasonably  good  multiple  choice  iteas. 
Using  a  writing  saaple  to  check  primarily  on  '•paragraph 
skills'*   is  a  case  of  using  the  wrong  testing  format. 
The  writing   test  should   allow  students  to  demonstrate 
control  of  paragraphs  as  naturally*-  occurring  units  in 
their  composing  process. 

Finally,   the  constraint  for  older  students — to 
write    100-503  rfords — whether   Included  in   a  test 
instruction  or  on  classroom  assignments,  encourages 
pointless  word  counting  that  typically  results  in  pad- 
ding rather   than   in  real   idea  development.     Where  the 
word  count  is  Intended   to  keep  students  from  writing 
too  much,  or   to  encourage  conciseness,  theeffort  to 
condense,   is — among  experienced  wr 1 ters — typically  a 
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revision  activity.   »nd  a  complex   one  at   that.  Most 
test-makers  w'no  include  the  word-count  restriction  do 
not  intend   a  close  counting  but  merely  aean   to  suggest 
the  general   scope  of  the   topic  —  how  auch  students 
should   attempt  to  do  in  the  time  available.     The  word 
limit  presjpposes  that  students  have  some  idea  of  wrtat 
a   3D0-vrford   paper   is  —  like  newspaper  columnists  who 
write  regularly  to   a  given   format.     A  more  natural 
limit  on  what  students  attempt   to  write   is  imposed  by 
th*  amount  of  time  provided.     Adding  any  other  con- 
straint on  length  may  lead   the  writer   to  perceive  word 
limit  as  being  in  conflict  or   incompatible  with  tne 
time  constraint. 

Although  this  informal  observation  about  the 
effects  of  specifying  number  of  words  remains  to  be 
tested   in   careful   research,   students  should   be  given 
enough  time  to  deal   with  the  prompt.     Tne  number  of 
words  would   then  be  determined  by  such  relevant  con- 
siderations as  how  mu3h  they  find   to  say  and   how  much 
they  are  able  to  revise,   in  the^tiae  available.  If 
they  are  not  writing  for  publication,   the  number  or 
words  attempted   is   probably  the  least  important  aspect 
of  the  composing  task.     In  a  testing  context  students 
Should   be  using  their  energies  and  time  on  concerns 
other   than  counting  words. 

The  Value  of  Focusing  Cues . 

One  of  the  most  helpful   constraints  that  can  be 
provided   in  writing   instructions   is  a   suggestion  to 
focus  on  one  aspect  of  a   larger   topic,   or  on  one  prob- 
leas  ^   or  on  one  or   two  examples  in   support  of  a  posi- 
tion.     Wnen   an  evaluator   tested   the  following  prompt, 
the   instructions  elicited  long  lists  of  general  com- 
plaints with  little  analysis  of  problems,  few  sugges- 
tions  for  solutions: 

PROMPT  48a 

Write  a  letter  to  your  principal.  Explain  to 
him  or  her  what  you  would  like  to  see  changed 
in   your  school  . 


The   tester  had   expected   students  to  write  as  if 
they  were  really  trying  to  bring  about  change.  The 
students,  on  the  other  hand,   interpreted  the  task  as  a 
general   sort  of  complaint  letter--a  "gripe  Iist«  — for 
anyone  who  will  listen.   A  revision  of  the  prompt  with 
focusing  constraint  provided   in   the  Instructions 
resulted   in  more   fully-developed  arguments: 
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PROMPT  i)8b 

Tnere   are  probably  several   things  you  would 
liKe  to  SS3  changed   in  your   school.  CHOOSE 
ONE  OF   THESE  and   write  a   letter   to  your  prin- 
cipal,  describing  or   explaining   the  probleti 
and   suggesting  what  he  or   she  night  do  to 
change  the  situation. 


Another  exaisple  of  the  value  of  providing  helpful 
focusing  cues  appeared   during   the  development  of  a  Col- 
lege  Board   English   Composition   Test   prompt.     One  year 
students  were  given  a  provocative  quotation  and 
instructed   to  write  about   it.  drawing  on  examples  from 
their   experience  or   their   reading.     The   prompt  gen- 
erated writing   that  represented    the   full   range  of  abil- 
ity  in   the   population  tested  when  it  was  adainistered 
and    scored.     The   following  year,   however,  when  the 
instructions  read   "Choose  one  or  Uio  examples  fro.n  your 
experience  or  reading."   one  reader  compared  results  and 
noted   clear   improvement.     The  earlier  version  haj  eli- 
cited writing  tnat  consisted  of  loosely  listed  anec- 
dotes.    Tne  latter  version  tended  to   elicit  decidedly 
more  coherent  writing,  with  students  focusing  their 
efforts  on   single,   telling   examples,   or  drawing  on  one 
or   two   pieces  of  literature   in   some  depth. 

One   word   of  caution   in  using  this  apparently  help- 
ful  strategy:   having  cued   students  to  concentrate  on 
one  or   two   ideas,    testers  should   leave  room   in  scoring 
for   the  exceptional  student  who.  working  at  a  different 
level  of  abstraction,  may  refer   to  many  examples,  but 
does  so  convincingly,   without  siaorificing  coherence  or 
depth. 

The  Effects  of  Speci  f ying  Audience . 

Specifying  a ud i en c e— tha t   is.  providing  writers 
with  an   imaginary  audience  other   than  the   test  read'er- 
-may  be  useful   in   some  testing  situations,  but  it  may 
cause  problems  In  others.     When  audience  is  specified 
in  a  writing  task  that  provides  a  full   rhetorical  con- 
text and   a  clear   purpose   for  writing,  the  effect  is  to 
create  a  more  realistic  writing  problem,  as  in  the  fol- 
lowing prompt   for   elementary  and  middle  school  or  high 
school  . 

PROHPT  49 

The   following   article  appeared   in   the  local 
newspaper' : 

KDDL  Begins  Survey: 

Wny  do  wj  remember  certain  people  from  our 
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pa«t  and  forget  others?  Television  station 
KDOL  Is  preparing  a  special  TV  show  on  per- 
sonalities people  remember  froa  their  elemen- 
tary school  years — a  classmate,  a  member  of 
the  school  staff,  a  family  member,  or  a 
neighborhood  friend.  Send  your  description 
to : 

Program  Producer 
KDOL 

Oakland,  CA  9^606 

Respond  to  the  article  by  choosing  someone 
who  made  an  unusually  strong  impression  on 
you.     Write  about  that  person,  explaining  why 
you  remember  him  or  her.     Do  your  best  work. 


Testers  seem  to  feel   that  merely  mentioning  an 
imaginary  audience  in  a  prompt  will  help  students  write 
with  more  enthusiasm  and  purpose   in  a  writing  exercise 
that  is  still  a  school  test,  one  whose  real  audience  is 
ultimately  the  teacher-scorer •     A  real  danger  lies  in 
the  injudicious  selection  of  an  audience  cue — for  exam- 
ple, "Write   for  a  friend. Following  the  lead  of  this 
instruction,  soaie  students  will  conscientiously  adopt  a 
tone  more  appropriate  to  writing   for  peers  than  for 
teachers.     Because  this  language  is  less  formal,  often 
less  articulate,  and  sometimes  less  comprehensible  to 
readers  than  tht  s tud en t • s     sc ho o 1  language,"  the  stu- 
dent who  is  mature  enough  to  vary  language  for  dif- 
ferent audiencea;  may  actually  be  penalized  during  the 
hoi  is  tic  scoring; .  » 

In  the  one  of  the  studies  conducted  under  this 
BAWP/NIE  Writing;  Assessment  Project,   the  prompt  had 
invited  students  to  think  of  a  first-time  experience 
and  to  write  about  this  experience  for  someone  who  is 
about   to  have  a  similar  experience .»»  Student-raters  of 
the  papers  said  that  certain  ones  sounded  like  English 
compositions  rather  than  like  the  frie:idly  letters  they 
pretended  to  be*     These  ^English  composition"  types 
were  marked  down  by  students,  but  they  received  top 
scores  from  teacher-raters,  while  other  papers  that 
received  low  scores  from  teacher-raters  received  high 
ratings  by  students  because  of  their  verisimilitude. 

"•MarTorT^rowhurst  (  1978)  has  researched  the  differences 
in  student  language  when  writing  for  peers  and  writing 
for  teachers  on  a  school  task.  In  some  modes,  students 
write  less  formally,  using  syntactically  less  mature 
language  when  assigned  to  write  "for  a  friend." 
»»3ee  Chapter  2,  Part  II. 
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Trial  responses  should  be  exaraintd  closely  for  signs 
that  students  are  using  less  mature,  less  explicit, 
less  effective  language  than  they  might  if  they  were 
addressing  their  actual   audience  of  unlcnown  adults. 

Other  effects  of  specifying  audience  may  aurprisa 
the  unwary  tester.   One  teacher  reports  that  he  has 
stopped  asking  students  to  write  to  pen  pals  because 
the  students  seemed  to  have  more  to  say  to  other  types 
of  audiences:  actors,  television  or  radio  characters, 
authors  of  books  and  articles  read   in  class,  members  of 
their   family,   friends  they  have  missed   since  their  last 
move,  makers  of  products  that  they  liked  or  didn't 
like,   other  students  in   the  school.     Consequently,  when 
this  same  teacher  was  asked   to  administer  the  following 
prompt  as  part  of  a  district  assessment,  his  students 
complained   that  It  was  "dumb,"  "boring,"  or  "phony," 
and   wrote  very  little,  giving  a  misleading  impression 
of  their  writing  abllliies: 

PROMPT  50 

Write   a   friendly  letter  to  a  pen  pal  In 
another  country,   telling  hia  or  her  about 
your  home  or  school. 

But   students  in  other  classes  who  had  had  little 
experience  with  writing   for  an  audience  did  not  protest 
the  prompt.     In  this  case,  the  teacher's  own  class 
required  a  more  sophisticated   choice  of  audience 
because  of  the  v er sa tl 1 i ty ^d e v el  oped  during  prior  writ- 
ing assignments. 

Other  attempts  to   specify  a  real  audience  also 
have  been  problematic.     For  instance,  some  assessments 
may  be  successfully  using  one  or  another  version  of  the 
"letter  to  the  principal"  prompt  discussed  earlier. 

PROMPT  48c 


There  Is  probably  at  least  one  thing  about 
your  school  that  you  would  like  to  see 
changed.     Write  a  letter  to  your  principal 
and  tell  him  about  the  problem.     Be  sure  to 
describe  the  situation  as  It  Is  now  and  tell 
why  It  bothers  you,  as  well  as  how  you  would 
like  to  change  It. 

This  version  of  the  prompt,   howaver,  had  to  be  aban- 
doned  after   pilot-testing  in  one  district  because  what 
the  students  wrote  Indicated   they  could  not  treat  the 
subject  seriously.     For  example,  teachers  In  one  school 
reported  derisive  laughter  and  direct  complaints,  even 
refusals  to  write.     Students  at  that  school  saw  their 


principal   a3  an  adversary,  uncaring  and  unwilling  to 
listen   to  r  eco.ianeni  ations .     At   a  second   school   in  the 
district!   whose   principal  was  especially  effective,  a 
homogeneous  and  coTiplacent  student  body  co.Taplained  that 
they  couldn't   think  of  a  problem  worth  writing  about. 
Tneir   awareness  of  the   triviality  of  the  problems  tney 
chose  was  reflected   in   tneir  writing. 

One  potentially  realistic  audience  specification 
tJhich  has  elicited   strong  essays  in  some  settings  is 
the  proposal   that  students  write  for  readers  of  the 
school  newspapar,  student  tnagazine,  or  yearbook.  Yet 
in  schools  wnere  there  is  no  student  press  or  where  it 
is  of  poor  quality,   good   writers  «ay  refuse   to  write  or 
may  write  badly  when  asked   to   imagine  they  are  writing 
for   this  audience* 

Initial   research  in   the  coaposing  processes  of 
college  writers   (Flower,    1979)   appears  to   suggest  that 
specifying  audiences  is  appropriate  when  the  writing 
task  is  to  comaunicate  for  business  and  social   or  pol- 
itical purposes.     In   the  real  world,  such  tasks  have 
audiences  whose  special   questions  or  needs  influence 
the  writer's  choice  of  what  to  say  and  how  to  say  it. 
In  Flower's  study,   good   writers  felt  more  confident  if 
they  were  given   a  particular   audience  when  asked  to 
write  on   an   informative  or  persuasive  topic.  Poor 
writers  paid   little   attention   to  audience  specification 
or   found   that  it  nade  the  task  harder.     In  this 
research  on  the  composing  process  of  good   and  poor 
adult  writers,   Linda  Flower   asked   subjects  to  write  for 
or  against  on   the  issu;e  of  abortion   for  re'aders  of  the 
Catholic  Weekly  or   the  Children's  Digest .     In  another 
study  she  had   teachers  and  engineers  describe  their 
jobs  for   readers  o  f ^  Seventeen  magazine.     For  self- 
expressive  or   literary  tasks,   specifying  audience  seetss 
to  make  little  difference,  as  if  students  recognize 
tnat  a  good  story  is  a  good   story — no  matter  who  the 
listener  or  reader.     (Woodworth  and  Keech,  1930.) 

In   the  Los  Gatos  Study*   (Keech)  ,   the  writing  tas*c 
provided   a  particular  audience  and   a  clear  rhetorical 
purpose  for  relating  personal   experiences*  This  task 
divided   students  into  those  who  simply  ignored  the 
audience  constraint,   thus  writing  the  kind  of  personal 
anecdote   the   test  maker   expected,   and   those  who 
attempted   to  make   a  real   transaction  of  the  piece,  giv« 
ing   themselves  a  much  more  difficult  task.     The  ver- 
sions of  the  prompt   tested  were: 


PfiOMPT  51a 


VEftSIOH   I   (implied  audience) 

Think  of  an  occasion  when  you  experienced 
something  tor   the  FIRST  TIHE.     It  may  nave 
been  soroething  you  later  caiie  to  do  more 
easily  or  something  you  now  take  for  granted; 
bjt  .  for  some  reason,   your  first  time  was 
memorable.     Write  about  this  "first  time" 
experience  so  that  your  reader  can  understand 
your  feelings  and  why  this  memory  nas  stayed 
with  you. 

PROMPT  51b 


VERSION   II   (specified   audience,  bracketed 
here  but  not  In  student  version) 

Tnink  of  an  occasion  when  you  experienced 
something  for  the  FIRST  TIME.     It  aay  have 
been  sonethlng  you  later  came  to  do  more 
easily  or  something  you  now  take   for  granted; 
bJt,  for  some  reason,  your  first  time  was 
aemorable.     {Imagine  that  you  ire  writing  to 
someone  who  has  Just  had  a  similar  experi- 
ence,  OR  to  someone  who   is  about  to  have  such 
an  experience.     Your  writing  might  help 
pre  pa  re  yourreader  for  the  experience,   or  it 
might'help  your  reader  understand   that  other 
people  have  gone  through  the  same  kind  of 
thing.} 


One  student,   composing  aloud   for  the  researcher, 
revealed  that  the  audience  constraints  in   the  second 
topic  meant,   for  her,  that  she  had  to  give  definite 
advice  to  a  friend  who  was  about  to  babysit  for  the 
first   time.     Sne  tackled  the  task  with  sophistication, 
bat  in  a  thirty-minute  test,  her  results  were  no  match 
for  the  easy  anecdotal  essays  of  students  who  had  ver- 
sion I,  or  who  treated  version  H  exactly  like  version 
I.     The  oral  protocol  of  this  student  reveals  compli- 
cated and  mature  strategies  for  writing  to  inform  in  a 
useful  way,  b'ut  the  time  provided  was  simply  not  ade- 
quate' for  her  to  finish  the  task  she  had  set  for  her- 
Self.     Other  writing  from  the  same  student  revealed  she 
wasperfectlycapableofdashlng  off  clever  anecdotal 
writing,  but  the  Instructions  in  this  case  had  seemed 
to  her  tp  call  for  something  more* 

-  There  are  no  simple  answers  to  the  question, 
••Should  the  writing  prompt  specify  audience?**  Strong 
reasons  may  be  advanced  for  creating  writing  tasks  that 
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specify  audience  and  rhetorical  purpose,  especially  ^ 
-Sen  the  tester  -ishes  to  call  for  persuasive  or  infor- 
»ative  writing,  but  pilot-testing  of  ^^^P^^'  i» 

essential  to  insure  that  students  are  helped  rather 

"  n  5  n  eJad  -hen  they  t.lce  the  P'-'^f^ '^^'i^rstiins- 
.-count.     The  test-maker  should  consider  two  questions. 
D^es  sp;cifylng  audience  add  information  that  l3  impor- 
tant to  the  writer  in  deciding  ho-  to  '•«>P°"J ^^''^f J« 
taSK?     Does  the  writing  task  really  '"PP^^*^/^*  '^'J^/^ 
writing  a  student  might  have  to  do  or  want  to  <lo./o'' 
an  audience  other  than  the  teacher-tester  who  will 
Ictually  be  scoring  the  papers?  .  If  the  answer  to 
titiler  of  these  questions  is  no.  then  the  audience 
specification  should  be  omitted. 
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CONCLUSION 

Tnere  Is  undoubtedly  a  great  deal  toore  ussful 
Information  about  what  makes  a  good  prompt  residing  in 
the  heads  of  experienced  essay  test  developers.  Also, 
there  are  growing  nuabers  of  "hoaeaade"  writing  tests 
being  generated  by  the  new  widespread   use  of  holistic 
writing  assessments.     But  local   test-makers  complain  of 
the  lack  of  accessible  information  on  how  to  make  good 
prompts  for  writers,   or  how  to  prepare  exacting 
instructions  that  direct  in  predictable  ways  the  act  of 
composing.     To  some  extent,  the  lack  of  available  "how 
to"  guides  is  a  result  of  the  fact  that  the  profession 
has  yet  to  agree  on  what  constitutes  a  "good"  easay 
test.     Writing  good  test  topics  remains  an  art,  not  a 
science,   and  criteria  for  judging  a  good  topic  may  vary 
according   to  an  evaluator's  purpose. 

It  Is  becoming  clear,  both  to  full  time  and  some- 
time evaluators,  that  more  needs  to  be  known  about  the 
effect  of  prompts  on   the  writer  being  tested.  Much 
ejclstlng   folk  wisdom  about  what  makes  a  good  prompt 
needs  to  be  tested.     Some  popular  practices  need  to  be 
carefully  qualified   before  being  further  promulgated. 
And,  as  with  objective  tests,  makers  of  these  direct 
measures  need   to  understand  the   seriousness  and  com- 
plexity of  their  task;  that  is,  the  extent  to  which  the 
prompts  they  give  will   Influence  the  outcome  of  of 
their  assessments.     They  need   to  be  alerted  to 
unresolved   problems  In  test  design  that  concern  all 
makers  of  direct  measures  of  writing  ability.  Educa- 
tors newly  charged  with  designing  writing  tests  that 
directly  measure  performance  can  be  helped  to  gain  this 
heightened  awareness  of  the  difficulty  of  their  task. 
They  can  also   perhaps  become  engaged   in  contributing 
ptw  Insights  to  test  design,  if  they  can  become 
acquainted  with  some  of  the  experience  and   Insight  of 
their  predecessors.     It  Is  in  the  hope  of  calling  for  a 
fuller  literature  on  the  problems  and  best  practlce.5  In 
test  design  for  direct  measures  of  writing  ability  that 
this  tentative  technical  report  is  offered. 


207 

189 


References 


ERIC 


Braddcck,  K.,  R.  Lloyd-Jones,  and  L.  Schcer,  EaifiSLCb  iD  WLill£E 
CCHB-Ciiticnt  National  Council  of  Teachers  of  English,  Urba- 
na,    Illinois  (1963). 

Carrp,  Gerald,  SfiQtfl  tlata  LQUUtX  WLXtins  aSSfiSSmtDt.  Bay  {rea 

Writing  Project,  University  of  California,  Berkeley  (197gj. 

Caplan,  Rebekah  and  Catharine  Keech,  SbQiiaS-SitUiDfl  a  tLfliDiDS 
ELDfltctD  tQ  b£le  StUricati  t£  SBCCifiC.  Bay  Area  Writing  Pro- 
ject, University  of  California,  Berkeley  (198C). 

Coffman,  W.E.,  -On  the  reliability  of  ratings  of  essay  exarnina- 

tions  in  English.-  EfiiSarch  id  tte  IcacblDa  Cf  EDSliSb.  vol. 
5(1),  pp.   2«^-37  (1971). 

Cooper,  Charles  R.  and  Lee  Ddell  .  EyaliiSliDg  «LUitig.  National 
Teachers  of  English,  Urbana,  Illinois  (1977). 

Cooper,  Charles  and  et  al..  ELitiDS  afeilitiSS  Ql  Lt2UULU' 

gsi!Dinesi  fL55tl!D£n  fil  iUtiX/E.ult£l2.   Unpublished  iranuscri pt, 
San  Diego  (1979). 

Diederich,  Paul,  Msaiyning  SLCalh  ID  EQ3liSb»  National  Council  of 
Teachers  of  English,  Urbana,  Illinois  (197<»). 

Flower,  Linda  and  John  Hayes,  "P robl em- sclv i ng  strategies  and  the 
writing  process,"  CcllCSfi  tDSliSb*  vol.  39(5),  pp.  %'»9-<»61 
(Decerrber  1977). 

Flower,  Linda,  -Writer-based  prose:     a  ccgn i t i ve  basi s  for  prcb- 
Tems   in  writing,"  £cU£££  EMllib,   vol.  «fl(l),  pp.  19-37 
(September  1979)  . 

Gibson,  Walker,  SfetlnS  iDJi  KCillDS.  Longnans,  Green  and  Coir.pany, 
Inc.,  Nek*  York  (1959). 

Gray,  Stephanie  and  Catharine  Keech,  WLUing  frgc  ciytQ  iDfQLter 
tiQDf  Bay  Area  Writing  Project,  University  of  California, 
Berkeley  (1980). 

Kirrie,  Marjorie,  "Prompt  writing  Is  not  Impromptu,"  HatiCDal 
ELiliny  ElqjwlI  tlcwsltltetf  vol.  i(3)  (May  1979). 

Mull  is,  1..  Ibc  BLLoaLy  Ualt  syslctD  foe  scqcIds  wtitlcs  i"^^. 

National  Assessment  of  Educational  Progress,  Denver,  Colore 
do  (197<i). 

Myers,  Miles,  k  ctfictduLt  iQL  wlUIds  assessmeDt  add  fcclistic 
^^Libflt  ERIC  Clearinghouse  on  Reading  and  Coirmun  icat  ton 
Skills  and  the  National  Council  of  Teachers  of  English,  Ur- 
bana,  Illinois  61801  (1980). 

20S 

190 


PcKell,  David,  Whal  CSD  1  wciie  afeQUtt  National  Council  of  Teach- 
ers of  EnglTsht  Urbana,   Illinois   (1981).     7.0C0  Topics  for 
High  School  Students 

Spandelf  Vicki  and  Richard  J»  StigginSf  DitSCt  !DaflSULfi5  at  WLltr 
ing  skill*  iSSUtS  and  atBliCfltiCDS,  Northwest  Regicnal  Edu- 
cational  Laboratory*  Portlandt  Oregcn  97204  (198C). 

Stahlf  Abraharrif  *The  structure  of  children's  compositions: 
developmental  and  ethnic  di  f  f  erence  s  t*  EeSftaCEtl  iD  ltl£ 

Itacbloa  of  Enaliiht  vol.  2t  pp.  156-164  (1977}. 

Whitet  Edwardt  CccBaLiSED.  aDSi  ECDtHailt  California  State  Univer- 
sities and  CollegeSt  Office  of  the  Chancellor  (1973). 

Woodworthf  Patrick  and  Catharine  Keech,   Ibft  MLltC  C££fiSlQn»  Bay 
Area  Writing  Projectt  University  of  Californidt  Berkeley 

(1980)  . 

Jeff&LiQD  CcuDty  wiitins  samtle  ccBQCt''  (1976). 

Iti£  asstsscanl  cl  the  aciting  cttfeLcaQcs  cf  CalifccDia  blab 

SCbCQl  i£niSL5.t  California  State  Departinent  of  Education 
(1977). 

Ih£  DaiUtE  and  CSaiUESOlfiDt  2f  CCtDECtCDCX  iO  EE13lish»  National 
Council  of  Teachers  of  English*  UrbanSf  Illinois  6ie01 

(1981)  . 


191 


ACKNOWLEDGEMENTS 


rie  are  grateful  to  the  nuaber  of  professionals  in 
testing  agencies,  the  university  professors  and 
researchers  in  writing,  and  the   Bay  Area  Writing  Pro- 
ject teacher— consultants  who  provided  written  comments 
on  an  earlier  version  of  this  report.     The  Bay  Area 
Writing  Project  teacher — consultant  critic  readers 
included  Ruby  Bernstein,  Kate  Blickhahn.  Mary  Frances 
Claggett,   Jean   Jensen.   Irv ing Pec kham  and  Bernard 
Tanner.     The  other  professional   critic  readers  included 
Evans  Alloway.   Cordon  Brossell,   Rex   Bro wn  ,  Cha r 1 es 
Cooper.   Kla  Davis.  Paul   Diederlch.  Edaund  Farrell. 
Sarah  Freedaan .   Ann  Huaes.   Ina  Hullis.   Bill  Robinson. 
Joe  Steele,  and  Edward  White.  .Q.^ 

We  are  also  grateful,  to  Marta  Cuiain  and  Ceorgie 
Kelra  Who  typed   the  several  versions  of  the  aanuscript 
and  who  patiently  tracked  down  elusive  reference  infor- 
■ation.     And   finally,  our  thanks  to  Leo  Ruth  and 
Deborah  Ruth  for  editorial  work  and  suggestions. 
♦ 


/■J  i  U 

192 


APPENDIX  A 


SUGGESTIONS  FOR  WRITING  ESSAY  QUESTIONS 

Gertrude  Conlan 
Educational   Testing  Service 
Princeton,  Mew  Jersey 


The  question  should  be  clear. 

Students  should  not  have  to  puzzle  over 
the   Instructions.     The   topic  Is  Intended 
to  test  the  ability  to  write  the  answer 
and  not  the  ability  to  guess  what  the 
test  naker  Intends.     Besides,  students 
have  only  a  limited  amount  of  time,  time  ^ 
that  should  be  spent  writing  and  not  | 
an alyzing  unnecessarily.  h 


The  question  should  be  as  brief  as  clarity  allows. 

Restatement  may  sometimes  be  necessary 
to  avoid  misunderstanding.     But,  then, 
perhaps  one  should  consider  whether  the 
restatement  should  be  used  without  the 
original  because  the  restatement  does 
not  need  additional  clarification. 


The  instructions  should  be  definite. 

Students  should  know  what  is  required. 
For  exaiiplt,  Discusa,  citing  specific 
examples  from  one  novel .  Or,  Pajr  at  ten-- 
tion  to  the  correct  form  of  the  business 
letter .     Or  ,  Be^  sure  to  use  complete 
sentences . 


Avoid  questions  requiring  only  a  yes  or  no  answer. 

Example:  Do  you  agree?     Where  does  the 
student  go  from  there? 


Average  students  should  be  able  to  write  average 
answers  to  the  questions,  and  yet  bright  students 
should  be  able  to  show  their  brightness. 
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A  good  topic  permits  the  ranking  of  all 
students  according  to  ability. 


The  vocabulary  used  and 
the  topic  should  not  bt 
ordinary  student  to  und 


the  concepts  expressed  in 
too  difficult  for  the 
erstand  ioaed lately. 


A  difficult  topic  distinguishes  only 
between  the  very  bright  and  the  rest  of 
the  population.     Besides,  difficult 
reading  changes  the  test  to  a  reading 
test . 


The  question  should  not  call  for  cliches  as 
answers. 

A  topic  worn  out  by  over  use  produces 
worn-out  responses.     On  the  other  hand, 
soae  good  questions  merely  twist 
cliches.     For  exaaple,  What'  s_  right  with 
television?     In  what  ways  are  teenagers 
more  conservative  than  the  over- 
thirties? 


The  question  Itself  should  provide  an  organizing 
principle  for  the  essay. 

Example:     Compare  and  contrast... 

Briefly  descr  ibe  . .  .and  then 

analyze  ..  . 

Discuss  your  answer  to  this  ques- 
tions , 

giving  the  reasons  for  your  answer 
and  citing  specific  examples  to 
support  those  reasons. 


The  question  should  not  elicit  responses  which 
interfere  with  either  the  writer's  or  the  read 
Judgment* 

Politics,  racial   issues,  and  other 
inflammatory  topics  ara  to  be  avoided. 
Also  to  be  avoided  art  topics  that  art 
dlshearteningly  dull.     For  txample,  tven 
a  seemingly  innocuous  topic  such  as. 
Who  has  rrad  the  greatest  influence  on 
ZZur  life*    cannot  be  scored  easily.  If 
uTPcanTTdate  writes  on  the  wrong  polit- 
ical  figure  (from  the  reader's  point  of 
view),  the  score  is  either  too  high  — 
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because  the  reader  is  making  up  for  his 
or  her  own  bias        or  too  low  because 
the  reader  has  succumbed  to  that  bias* 
On  the  other  hand,  the  fifth  essay  on 
the  greatness  of  the  basketball  coach  is 
not  scored  on  the  same  standard  as  the 
first.     Readers  are  human;  they  do 
become  bored • 


10.     The  question  writer  should  write  out  the  answer 

expected  and  determine  whether  the  question  really 
calls  for  that  answer.     The  question  writer  should 
also  try  to  answer   the  question  in  the  allotted 
time.   Just  to  see  whether  it  is  humanly  possible 
to  do  so.     The  question  should  be  revised  in  the 
light  of  any  discoveries  made. 


1979 
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SOME  CONSIDERJITIONS   IN  JUDGING 
THE  >*ORTH  OF  AN  ESSAY  TOPIC 


Gertrude  Conlan 


Do  the  candidates  seea  Interested  In  the  topic? 
Do  they  all  have  something  to  »ay  in  response  to 
i  t? 

Can   they  find  specific  examples  to  use? 

Are   the  responses  unduly  emotional  or  patriotic  or 
religious  or  In  some  other  way  difficult  for 
readers  to  Judge  fairly? 

Can  readers  make  Judgments  easily  between  essays 

at  different  levels  of  competence? 

Are  all  the  essays  alike  —  mediocre? 

Dots  the  topic  encourage  cliche-ridden  or  "canned" 

5"r;srtopic  mvit. 

at  what  attitude  the  readers  want  them  to  take 

ro:r?t'J:.:rtriters  trymg  to  take  .ll^PO"^^/" 
positions  because  they  fear  their  own  views,  may 
meet  with  disapproval? 

Is  the  topic  confusing  in  some  way? 

Are   the  requirements  of  the  topic  ambiguous? 

Is  the  topic   subject  to  several   accurate  Interpre- 

itii^JSese  create  problems  for  scorers,  by  making 

comparison  of  quality  difficult? 

Are  the  candidates  misreading  or  misco«»str ulng 

some  word  or  phrase   in  the  topic? 

Are  candidates  following  the  requirements  of  the 

is'thi  -ode  of  writing  the  °  •'^J  %  "Vj"  ^ " 

ing  with  the  -ode  the  topic  is  meant  to  produce? 

How  will  readers  react  to  the  topic? 
Are  the  responses  Just  plain  boring? 
Can  Judgments  be  made  quickly  on  reasonable 

irTSuh  scores  being  a-arded  for  writing  ability 
.nS  not  for  extraneous  matters  like  sophisticated 
literary  allusion*  or  unique  examples? 

How  long  are  the  essays?  1^^^1.7 
Has  too  much  time  been  allowed  or  too  little? 

Do  the  essays  lack  a  starting  point? 

Do  the  essays  lack  organization? 

IS  the  topic  at  fault  for  falling  to  suggest  a 
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method  of  organization? 

Is  the  topic  at  fault  for  over-suggesting  a  method 
of  organization  —  no  variety? 

Would  you  be  willing  to  read  essays  on  this  topic 
for  several  days  in  succession? 
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PROMPT   IWDEX  «ITH  SOURCES* 
PROMPT  la 

Think  about  the  word  tre_*s.     Write  abo^t  -hat  the 
word  suggests  to  you.     (Derived   froa  topic  32 
below,  unplloted  exac»ple.) 

PROMPT  lb 

Think  about  the  word  trees.     Write  a  jftory  in 
which  trees  play  an  important  part.   /^^J  "'^ 
real  experience  or  an  laiaginary  story.     ^f""  ""^ 
even  write  as  if  you  were  .  tree.     Be  sure  that 
trees  play  soae  part  in  your  story.     (Adapted  by 
C.   Keech,  unplloted  «ixaiaple.) 

PROMPT  10 

Think  about  the  word  trees.     Write         ""J  ^"^^ 
Which  you  explain  one  of  several  P°  f  ,  I'' 

this  word  «ay  suggest  to  you.  You  may  '^""'^ 
a  special  kind  of  tree,  a  special  use  ^j;"' 
people's  attitudes  toward  trees,  or  ^oo^^hlng  else 
that  int-rests  you  about  trees.     Focus  on  one 
IdeL-do  not  attempt  to  list  all   ^he  things  trees 
can  -ean.     (Adapted  by  C   Keech.  unplloted  exam- 
ple. ) 

PROMPT  2a 

Write  about  Deceaber  25.  1973* 


PROMPT  2b 


Write  about  an  experience  you  had  during  .  family 

holiday  which  was  particularly  aeaorable.  It  may 

Se  hajpy  or  sad.  but  should  be  a  time  when  you  had 
strong  feelings.     ^ ^y ^C. ^Keech . ) 


TH-^tm-     The   prompts  ln"th7  following  list  have 
Je       in     thf*     Report  to  illustrate  -'"f fJ^/^Si 

:rthrur^%^rntis;rario-  ^^f- isr ^^'^ 

these  are  marked  by  the  null,  sign  0. 
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Write  about  a  place  that  is  special  to  you.  Help 
your  reader  understand  its  peculiar   interest  or 
significance.     The   place  you  choose  way  be  of  any 
kind— for  example,  a  personal  haunt,  a  historical 
spot,  or  an  institution  or  a  place  you  reaeraber 
vividly  frod  tarly  childhood.     (Variation  of 
pronpt  used  in  the   Sir  Francis  Drake  High  School, 
San   Anselao,   California  school-wide  assessment, 
1958.) 

PROMPT  4 

Write  about  an  object  you  are  especially  attached 
to,  something  that  has  deep  personal  meaning  for 
you,  soaethlng  that  has  become  a  part  of  your 
life.     You  night  want  to  consider  the  way  you 
discovered  it,  the  way  it  came  into  your  life,  the 
way  it  has  taken  on  meaning  through  time.  (Varia- 
tion of  prompt  used   In  the  Sir  Francis  Drake  High 
School,   San   Anselao,   California,  school-wide 
assessment,  1972.) 

PROMPT  5 

Tell  about  soaething  you  enjoyed  doing  as  a  child. 
(Caplan   and  Keech.   p.  48,   used   in  research  study.) 

PROMPT  6 

Tell  about  an  experience   from  which  you  learned  » 
lesson.     (Caplan  and  Keech,  p.   47,  used  in 
research  study.) 

PROMPT  7 

Think  of  someone  you  admire  or  resent.     (Mot  your 
parents.)   Write  about  this  person  and  include 
incidents  that  led   you  to  admire  or  resent  this 
person,   so  the  reader  can  understand  why  you  feel 
the  way  you  do.     (Las  Lo.ias  High  School,  Walnut 
Crtek,   California,    1977  school-wide  assessment. 
Ed  i  ted  V  er  si  on . ) 

PROMPT  8 

Describe  a  change   that  has  happened   in  you 
recently  or  an  tvent  that  happened   a  long  time  ago 
but  still  affects  you.     TtU  of  the  events  that 
surrounded   it  and   caused   the  change  in  you  so  your 
reader  can  understand  what  is  now  different  about 
you  and  why.     (Las   Lomas  High  School,  Walnut 
Creek,   California,    1978  school-wide  assessment. 
Edited  version.) 
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PRD.iPT  9 


Each  of  us  assu.nes  a  personality  Co   satisfy  each 
group   into  wnich  fta  enter;   i.e.,   we  do  not  act  at 
hotne  precisely  as   we  act  with  the  gang,   in  the 
classrooni,  or  before  an  eojployer.     (College  Board 
Examination  relea:,ad  topic  —  adopted  version.) 

PROMPT  10 

W'-ite  about  your  fav^orite  place*  (A  reduction  of 
Topic  3.^ 


PHCMPT  11 

rfrite  about  y^ur  favorite  object.  (A  reduction  of 
Topic  ^ • ) 

PROMPT  12a 

Tnink  of  an  occasion  when  you  experienced  some- 
thing for  the  first  time.     It  may  be  something  you 
later  caae  to   take  for  granted,  but   for   soiie  rea- 
son »  the   first  time  was  memorable.     Write  about 
your  experience  so  that  your   reader  understands 
your   feelings  at  the  tiiie  and   why  it  stays  in  your 
meusory.      (C.    Keecn,  one  of  two  versions  used  in 
Los  Gatos  High  School,   California  experimental 
stuiy .    1930. ) 

PROMPT  12b 

Ttiink  of  an  occasion  when  you  experienced  sorne- 
tning   for  tne  first  ti^e* 

CU£3: 

Toere  is  a  book  called  The  First  Time,   in  which 
fa-nous   people  describe   their   first   kiss  or  first 
love  affair.     But  there  are  many  other  important 
First  Times  in  our  lives*   from  the   first  time  we 
tied  our  own   shoes,  or  went  to  school  or  camp,  to 
our   first  time  in  deep  water,   or   first  big  gane  or 
first   speech  in   front  of  a  group.     Some  of  these 
occasions  are  first  and  only  times.     Others  are 
the  first  of  amy  times,  but   for  some  reason  the 
first  time  was  memorable. 

INSTRUCTI0H3: 

Cnoose  one  of  these  occasions  that  stands  out  in 
memory  and  write  about   it  so  that  your  reader  can 
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understand  your  eiperi-^nce  and  why  it  was  memor- 
able.    ( C.   Keech.  un piloted  eiaaple. ) 

PHOrtPT  12c 

Think  of  an  occasion  Mhen  you  experienced  some- 
thing for  the  FIRST  TIME.     It'-^aay  have  been  some- 
thing you  later  cane  to  do  acre  easily  or  soae- 
thing  you  now  take  for  granted;  but,  for  some  rea- 
son, your  first  time  was  jiemorable.     Imagine  that 
you  are  writing  to  soaeone  who  has  Just  had  a 
similar  experience,   OR  to  someone  who  is  about  to 
have  such  an  experience.     Your  writing  might  help 
prepare  your  reader   for  the  experience,  or  it 
aight  help  your  reader  understand  that  other  peo- 
ple have  gon?  through  the  same  kind  of  thing.  (C. 
Keech,  one  of  two  versions  used  In  Los  Gatos  High 
School,  California  experimental  Study,  1930.) 

PROMPT  13 

Compare  two  stores  you  know  whib^  sell  slailar 
iteas.     Consider  such  things  as  prices,  aerchan- 
dise,  service,  atmosphere,  location,,  or  other  spe- 
cial  features.    Hftich  of  the  two  storVs  y9U 
prefer?  Ex  plain' w!iy .     (Gray  and  Keech,  p.  26,  used 
in  research  study. ) 

PROMPT  '  .  ♦ 

Coapare  two  actual   holidays  or  special  ev.ents  you 
celebrate  in  your   faaily  (such  as  Thanksgiving; 
Passover,  Christmas,  the  Four.th  of ,  July ,  your  . 
birthday,  etc.)  Consider  *uch  things  as  food, 
decorations,  activities,  behaviors,  atmcTsphere^  or 
other  special  features.     Which  of  the  two  do. you 
prefer?     Explain  why.     (Gray  and  Keech,:  p.  26, 
used  in  research  study.)  .  . 

PROMPT  15a 

You  have  met  a  man  from  outer  space  who"  ha  a  landed- 
on  earth  near  your  school  playground.     He  can 
understand  English,  but  he  does  not  know  an'ything 
•bout  8chool«her«  on  earth.     Describe  your  schpol 
for  him.     TeU  him  about  your  school  building. 
TeTl~how  the  school  building  looks  on  the  o"t»t'^* 
and  the  inside.     Tell  him  about  your  teacher  and 
77ur~cTassmates.     (italics  added)   ( Cal  i  f  oj-iti  a 
State  Board  of  Education  Handbook  on  Assejsaent,^ 

197  9.) 
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PROMPT  15b 


Imagine  a  friend  of  yours   is  about  to  move  to 
to-n.     Your   friend's  name  is  Pat.     Pat  will  be 
aoinn  to   your   school.     Below,  write  a  letter  to 
?at.  describing  one  or  two  things  you  think  a  new 
student  suould   know.     You  night  tell  about  what 
you  do  at  lunch  time,  or  describe  the  gaites  you 
play.     You  night  describe  your   favorite  part  of 
the  classroo«  or  what  your  teashar  allows  you  to 
do  in  class  activities.     ( C   Keech  variation  of 
Topic   15a.  unpiloted.) 

PROMPT  16a 

W-ite  about  your  favorite  place.     Maae  the  pLace 
and  tell  where  it  is.     Tell  how  often  you  go 
there.     Wny  is  this  place  special  to  you?     Ho«  do 
you  feel  when  you  are  there?     Wnat  are  soae 
'sights,   sounds,  colors  and  s.ells  in  your  fjvorit. 
Dlace?     (San   Mateo   City  Eleaentary  School  Dis- 
trict.  California.   Courtesy  of   Joanne  Graham.) 


PROMPT  16b 

Everyone  has  a  special  place  where  they  like  to 
ao      »<rite  about  your  favorite  place  to  go. 
i:;crtbe  the   place  so  yo.ur  reader  can  -^/^ 
it  is  really  like.     Tell   about  what  you  do  there, 
and  how  you  feel  when  you  are  there.     Help  your 
reader   see  Why  this  place  is  special.   (Revision  of 
16a.) 

PROrtPT  17 

rfrite  a  story  that  begins:     'One  day   I  ^^""J 
.agio  machine.'    (Middle   »  =       ^  '  l^s-ss! 

school   of  the  Hague.   1975.  unpiloted   in  assess- 

men  t . ) 

PROMPT  18 

This  morning  on  my  way  to  school.   I/^-  ' 
black  hole.     (C.   Keech.   classroom  assignment, 
unpiloted  in  assessment.) 

PROMPT  19 

The  last  thing  I  remember  was  the  ship 
J^ople  screaming  and  running   for  lifeboats.  Then 
I  blacked  out.     Wnen  I  came  to.   I  was...  (C. 
Keech.  classroom  assignment.) 
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PROMPT  20a 


Dear   rflsh  F.airy,  ^ 
I  wish  I  had    *   .  • 

INSTRUCTIONS: 

The  teacher  gives  the   following  instructions 
orally: 

**Pretend   I  am  the  Wish  Fairy  who  can  aake  your 
wishes  come  true.     I  want  you  to  write  me  a  letter 
to  tell  me  what  you  wish  for.     Think  of  several 
things  you  wish  for.   (pause)  Now  then,  choose  the 
one  thing  you  wish  for   the  very  most. 

Tell  me  what  it  is  by  finishing  the  sentence,  I 
wish  I  had..."  What  do  you  wish  you  had?  After 
you  write  down  its  name,  then  go  on  writing  and 
tell  me  all  about  it.     What  is  it  like?  Write 
down  everything  you  think  and  feel  about  your 
wish.     (Thanks  to  an  unknown  elementary  school 
teasher  who  presented  it  at  a  California  Associa-> 
tlon  of  Teachers  of  English  Conference.  Unpiloted 
in  assessment.) 

PROMPT  20b 

''I  wish  I  were ..." 
** I  wish  I  could  . .  •  ** 
*^I  wish  I  knew. . 
(Variations  in  common  use.) 

PROMPT  21 

Write  about  an  event  you  wish  you  had  witnessed  or 
could  witness.     Tne  event  can  be  real  or  imagined; 
the  time  of  the  event  can*  be  past,  present,  or 
future.     Make  it  clear  why  the  event  is  signifi^^ 
cant  to  you.  (Variation  of  prompt  used  in  Sir 
Francit  Drake  High  School,  San  Anselmo,  California 
annual  assessment,  1973.) 

PROMPT  22 

If  you  had  to  choose  to  be  something  other  than  a 
human  being,  what  plant  or  animal  or  other  form 
would  you  choose?     In  your  writing,  give  your 
reader  some  idea  of  what  you  think  it  would  be 
like  to  be  that  form,  and  of  why  you  chose  it. 
(Variation  of  prompt  used  in  Sir  Francis  Drake 
High  School,  San  Anselmo,  California  annual 
assessment ,   197^* ) 
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PROMPT  23 


If  ^ou  couia  change  placts  wtth  someone  else,  who 
would  it  be?     The  person  you  -rite  about  can  be 
living,  dead,   irawn  from  past  or  present,  from 
books,  films,  etc.,  or  from  your  own  imagination. 
In  your  writing  give  your  reader   some   idea  of  what 
it  would  be  like  to  be  that  person,  and  of  why 
that  life  appeals  to  you.     (Variation  of  prompt 
used  in   Sir  Francis  Drake  High  School,  San 
Anselmo,   California  annual   assessment,  1975.) 

PROrtPT  21 

What  would  you  do  if  you  had  a  million  dollars? 
(Topic   in  common  use.) 

PROMPT  25 

Mnile  visiting  in   your  grandmother's  house  you 
find  a  large  trunk.     Propped  on  a  chair  beside  the 
trunk  is  a  card;  on  it  in  your  grandmother's 
handwriting  are  the  words:  "To  ay  only  grandchild: 
Inside  this  trunk  you  will  find  what  you  have 
always  wanted."  Describe  the  gift  and  how  you  will 
use  it.     (Santa  Clara  County  Field  Test  of 
Prompts .    1931,  pT  6.  ) 

PROMPT  26 

Imagine  that  you  are  trapped  in  one  of  the  follow- 
ing situations  and  you  cannot  escape  for  ^^''^fal 
hours.     Tell  what  you  do  to  escape  or  to  pass  the 
time  until'you  are  rescued: 

A.  an  elevator 

B.  an  abandoned  mine 

C.  a  locked  room 

D.  a  life-boat 


(Santa  Clara,  County  Field  Test  of  Proa£ts,  1981 
pT  5)  " 


PROMPT  27 


Imagine  that  you  have  just  landed  on  the  moon. 
Mrite  about  what  happens  after  you  climb  out  of 
your  space  ship  and  set  foot  on  the  surface  of  the 
moon.     (Suggested  by  Michael  Scriven,  piloted.) 

PROMPT  28 
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Pretend  you  rfpite  the  idvioe  column  in  our  school 
newspaper.      5fou  must  answer   the   following  letter: 

Dear   Happy   Hel per  : 

I  have  a  little  sister  who  always  hits  me 
when   I  have  a  toy  or  a  cookie  she  wants.  My 
grandnother   says   I  can't  hit  her  back.     Sne   says  I 
should   share,   but  I  get  tired  of  sharing.     Wnat  do 
you  think   I  should  do?     (C,   Keech,  unpiloted  in 
assessinen  t .  ) 

Signed  , 

In  Trouble 

PROMPT  29 

Explain  your  agreement  or  disagreeaent  with  the 
following  statement:   The  history  of  woaen  in  our 
country  is  the  history  of  oppression.  (Source 
lost.) 


PROMPT  30a 

"Men   are  superior  to  woien.**   Agree  or  disagree 
with  this  statement,   giving   reasons  for  your  opin- 


ion 


PROMPT  30b 

''Women  are  superior  to  men."  Agree  or  disagree 
with  the  statement,  giving  reasons  for  your  opin--- 
ion  « 


PROMPT  30c 

**Women  are  better  off  thin  men   in  today's 
society."  Agree  or  disagree  with  the  statement 
giving  reasons  for  your  opinion. 


PROMPT  30d 

Explain  what  is  wrong  with  the  following  argument 
"It*s  pretty  clear  that  on  the  whole,  women  are 
superior  to  men.     In  the  physical  dimension,  the 
best  evidence  for  this  is  simply  that  women  live 
longer ,  and  by  a  substantial  number  of  years. 
Other  evidence  is  that  women  ar*  virtually  immune 
to  several  diseases  that  are  fatal  for  many  men; 
and  on  the  mental  side,  there  is  the  simple  fact 
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that  girls,  on  the  average,  do  better  at  school 
than  boys."   (Michael  Scriven  for   Carnegie  Evalua- 
tion  of  Bay  Area  Writing  Project.) 

PROMPT  31a 

At  the  place  where  you  rfork,  a  woaan  has  just  quit 
her  job,  leaving  vacant  the  company's  only  execu- 
tive position  ever  held  by  a  feaale.     Tne  Board  of 
directors  has  stated  their  preference  that  a  wo.Dan 
replace  her  in  order  to   fulfill   an  Affirmative 
Action  quota.     As  a  member  of  the  Hiring  Commit- 
tee»  it  is  your  job  to  help  choose  a  successor  to 
the  post. 

The  only  woman  who  has  applied   for  the  job  seems 
competent  and  meets  the  written  qualifications  for 
the  job  but  she  is  clearly  less  qualified  than 
both  of  the  men   she  is  competing  with.     Members  of 
the  Committee  disagree  about  what  should  bt  done: 
some  say  hiring  a  woman   is  absolutely  necessary 
for  breaking  down  employment  discrimination;  oth«- 
ers  say  hiring  a  less  qualified  person  would  be 
foolish  as  well  as  unfair  to  those  working  under 
the  new  executive. 

To  have  a  full  hearing  of  all  views  on  this  criti- 
cal  issue,  the  Hiring  Committee  has  asked  each 
mir.in>tY  to  prepare  a  carefully  written  statement  to 
be  distributed  in  advance  of  a  meeting  to  discuss 
the  issue.     Write  a  statement  which  represents 
your  position  in  the  ma t ter ,  m aki ng  it  as  logical 
and  persuasive  as  possible.     Your  writing  task  is 
to  persuade  the  Committee  to  adopt  your  own  view 
and  to  vote  on  the  job  candidates  in  accord ance 
with  your   view.     (Cnarles  Cooler,  Wr i  ting  Abili- 
ties of  Regularly-  Admi  t ted  Fr eshmen t  at 
SUNY/Buf f alo ,    197  9- ) 

PROMPT  31b 

Argue  pro  or  con  affirmative  action  policies  in 
admissions  or  employment.     TC    Keech  variation  of 
31a  .) 


PROMPT  32 

Consider  the  words  trees  and  plastic .     Write  a 
paragraph  in  which  you  discuss  some  aspect(s)  of 
what  the  words  suggest   to  you.     (Marjorie  Kirrie. 
Portland   University,  Oregon.) 
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PROMPT  33 


Even  those  of  us  who  -lay  conform  outwardly  have 
our  private,  personal  forms  of  rebellion.  Some 
rebel  against  society's  conventions,  some  against 
family  values*  and  others  against  the  attitudes  of 
peers.     We  might  dress  in  unconventional  tfays, 
reject  our  family^s  traditions,  or  disagree  <#ith 
friends*   political  beliefs.     Some  of  us  may  simply 
rebel  against  popular  fads.     Hany  of  us  are  not 
interested  in  Jogging  five  miles  a  day,  eating 
natural   foods,  or  wearing  designer  Jeans.  (Jo 
Keroes,   San  Francisco  State  University.) 


PROMPT  34 

"Man  is  a  creature  of  habit." 

Do  our  habits  make  us  more  stable,  more  efficient, 
or  Just  duller?     Discuss.     Support  your  arguments 
with  one  or  two  specific   illustrations  from  your 
reading  or  experience.     (Adapted   from  released 
Coll  eg e  Entrance  Examination  Board  Topics ,  1963- 

1 955.  ) 

PROMPT  35 

**Opposition  is  indispensable.     A  person  will  often 
learn  more  from  the  people  who  disagree  with  him 
or   her  than  from  those  who  agree. ^ 

Do  you  agree  with  this  observation?     Have  you  pro- 
fited more  from  people  whose  views  were  similar  to 
yours  or   from  people  whose  views  opposed  your  own? 
Discuss,  using  one  or  two  specific  examples  to 
support  your  position.     (Adapted  from  released 
College  Entrance  Examination  Board   Topic  s ,   195  3- 

1956.  ) 

PROMPT  36 

Not  all   inventions  have  been  good  for  humanity. 
Name  one  invention  we  would  be  better  off  without, 
and  make   it  clear  why.     You  may  do  this  writing  as 
an  essay.  Journal,   letter,  story,  or  other  form. 
(Variation  of  prompt  used  in  Sir  Francis  Drake 
High  School,  San  Anselmo,  California  annual 
assessment,   1976. ) 
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PROMPT  37 


Imagine  that  a  small  group  of  people  will 
to  colonize  a  new  planet.     Food,  clothing,  shelter 
and  transportation  have  been  provided   for.  You 
are  among  those  asked   to  select  a  few  additional 
things  to  be  sent  along  in  the  limited  space 
available  in  the  ship.     ^^nat  one  item  would  you 
recommend,  and  why?     You  may  write  your  recommen- 
dation in  the  form  of  a  story,  a  dialogue,  a 
letter,  a  speech,  an  essay  or  other  form.  (Sir 
Francis  Drake  High  School.   San  Anselmo,  California 
annual  assessment.) 

PROMPT  33 

Describe  your  favorite   ^  (person,  food. 

object)   so  that  your  reader  understand  why 

is  your  favorite .     (Commonly  used  topic.) 

PROMPT  39 

My  Home:     Write  a  description  of  your   home,  its 
rooms  and  their  contents  in  such  a  way  that  some- 
one who  has  not  visited  it  can  form  an  idea  of  ths 
way  it  looks.     (Abraham  Stahl .  Research  In  the 
Teaching  of  English.    1977.   suggested  by  Leo  Ruth.) 

•  PROMPT  40 

"Many  observers  of  our  society  claim   that  modern 
man.  immersed  in  materialism,  is  '.owned  by  his 
objects.'     Yet  many  of  us  have  objects  that  we 
treasure  not  Jqst  for  thetr  material   value ^bu.  for 
a  variety  of  other  reasons.     Assignment:  Describe 
one  or  more  objects  which     are  Important  to  you. 
Explain  What  values  they  represent  and  comment  on 
those  values.-   (Edward  M.   White.  The  California 
Stat-  University  and  Colleges  Freshman  English 
Equivalency  Examination.    1975— sample  essay  ques- 
tion.) 

PROMPT  11 

"Find  a  church  in  your  neighborhood  that  aeems  to 
you  an  interesting  building.     Describe  what  you 
see  on  its  facade  in  detail  and   in  such  a  style 
that  your  reader  sets  it  as  interesting  too. 
(Walker  Gibson.  SeeU^i  and  Writinfi.    1959.  class- 
room assignment, "unpiloted  in  assessment.) 

PROMPT  U2a 

-Describe  what  you  see.-     (Accompanying  •  P'J^J" 
and  diagram  of  an  automobile  accident.)   (Based  on 
a  1976  California  State  Assessment  topic.) 
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Imaglnt  that  a  small  group  of  people  will  be  sent 
to  colonize  a  new  planet.     Food,  clothing,  shelter 
and   transportation  have  been  provided  for.  You 
are  among  those  asked  to  select  a  few  additional 
things  to  be  sent  along  in  the  limited  space 
available  In  the  ship.     tfhat  one  Item  would  you 
rtcomaend,  and  why?     Tou  may  write  your  recommen- 
dation In  the  form  of  a  story,  a  dialogue,  a 
letter,  a  speech,  an  essay  or  other  form.  (Sir 
Francis  Drake  High  School,   San   Anselmo,  California 
annual  assessment.) 

PROMPT  38 

Describe  your  favorite    (person,  food, 

object)  so  that  your  reader  wi 11  understand  why  it 
I s  your  favorite .     rCommonly  used  topic . )  ' 

PHOMPT  39' 

My  Home:     Write  a  description  of  your  home,  its 
rooms  and  their  contents  in  such  a  way  that  some- 
one who  has  not  visited  it  can   form  an  Idea  of  the 
way  it  looks,     (Abraham  Stahl,  Research  in  the 
Teaching  of  English,    1977.  suggested  by  Leo  Ruth.) 

PROMPT  MO 

••Many  observers  of  our  society  claim  that  modern 
man,  iicmersed  in  materialism,  is   *owned  by  his 
objects.*     Yet  many  of  us  have  objects  that  we 
treasure  not  Just  for  their  material  value  but  for 
a  variety  of  other  reasons.     Assignment:  Describe 
one  or  more  objects  which    are  important  to  you. 
Explain  what  values  they  represent  and  comment  on 
those  values,"  (Edward  M.  White,  The  California 
State  University  and  Colleges  Freshman  English 
Equivalency  Examination,   1975 — sample  essay  ques- 
tion.) 

PROMPT  41 

••Find  a  church  in  your  neighborhood  that  seems  to 
you  an   interesting  building.     Describe  what  you 
see  on  Its  facade  in  detail  and  In  such  a  style 
that  your  reader  sees  It  as  Interesting  too.'* 
(Walker  Gibson,  Seeing  and  Writing,    1959,  class- 
room assignment,  un pile  ted  In  assessment.) 

PROMPT  4|2a 

"Describe  what  you  see.'^     (Accompanying  a  photo 
and  diagram  of  an  automobile  accident.)   (Based  on 
a   1976  Gal I fo  rnla  State  Assessment  topic.) 
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PROMPT  ^2b 


Study  the  accident  in  the  photo,  usin^  the  diagram 
to  help  you  gain  a  clear  understanding  of  the 
situation.     Describe  the  accident,  reporting 
details  accurately  so  that  a  jury  or  insurance 
company  can  .-nake  a  fair  decision  about  who  is  to 
blame.     (Variation  of  Topic  42a.     Idea   froa  Beth 
Br  enn  em  an  . ) 

PROMPT  43 

The   following  ad v er ti Sf men t  appeared   in   the  loc2l 
paper  : 

SUMMER  ErtPLOYMENr 

Earn  i  Learn 

On-the-job  training  for  future  positions  In: 

Medical  Services  Aide  at  Highland  Hospital 
Ticket  taker  at  local  theatre 
Grocery  clerk  at  local  supermarket 
Plumbing  Assistant  for  plumbing  company 
Assistant  to  the  manager  of  a  oircus 

Earn  $3.50  or  more  per   hour  while  you  learn  a 
valuable  skill. 

Send  letter  of  application  to: 

Ms.   Laura  Jones,   Opportunity  Training  Center 

2212  Fruitvale  Ave.,   Oakland,   California  9^231 

Pretend  that  you  are  applying   for  one  of  the  jobs 
listed.     Write  a  letter  explaining  why  you  should 
be  hired.     You  should  give  information  about  your- 
self.    The  xt  formation  may  bt  actual   facts,  made- 
up   information,  or  a  combination  of  fact  and  fic- 
tion.    The  letter   should  help  you  get  the  job. 
(Oakland   School   District  Assessment,   Oakland,  Cal- 
ifornia.) 

PROMPT  44 

Describe  your  school.     Tell  how  it  looks  from  the 
outside.     Tell  how  your  classroom  looks.     You  may 
describe  what  happens  In  the  hallways  at  different 
times  of  day,  or  you  may  describe  the  lunchroom  or 
playground.     If  you  like,  you  may  describe  the 
people  in  your  school.     (Source  unknown.) 
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0         Look  carefully  at  the  picture* 


These  kid 
turned  bo 
in  the  pi 
are  sotaeo 
Tell  what 
Write  as 
friend,  i 
Help  your 
is  provid 
Asse  sssen 
05-W-02, 


5  are  having  fun  Jumping  on  the 
at.  Inagine  you  are  one  of  the 
cture«  Or  If  you  wish,  imagine 
oe  standing  nearby  watching  the 
is  going  on  as  he  or  she  would 
If  you  were  telling  this  to  a 
n  a  way  that  ex pr esses  strong 


over- 
children 
that  ycu 
children 
tell  it. 
good 

feelings  • 


friend  feel  the  experience  too.  Space 
ed  on  the  next  three  pages.  (Hational 
t  of  Educational   Prcgress,   Report  No. 
Expressive  Writing ,  1976.) 

PROMPT  46a 


Look  at  the  picture  above, 
any  way  you  wish  by  writing 
following  ways:     as  a  story 
ters  been  doing?  What  could 


You  can  react  to  it  in 
about  it  in  one  of  the 
(What  have  the  charac- 
happen  next?) ,  as  a 


description,  or  as  an  opinion.  (Jefferson  County, 
Colorado,   Writing  Sample  Report ,  1976.) 

PROMPT  M6b 

Look  at  the  picture  above.  Let  it  give  you  ideas 
or  create  a  mood.  Then  use  the  ideas  or  the  mood 
to  write  a  story,  a  memory  of  something  that  hap- 
pened to  you,  or  an  essay  or  argument  based  on 
some  strong  feeling  inspired  by  the  picture.  (C« 
Keech  revision  of  Topic  46a.) 

PROMPT  47 

'*Na  mun  is  an  Island,  entire  of  himselfo  Every 
•  an   is  a  piece  of  the  Continent,  a  part  of  the 
main . .  .** 


John  Donne 


This  topic  should  lead  you  to  think  about  the  pos- 
sible meaning  of  the  quotation.     You  may  build 
upon  the  idea  by  supporting  it,  or  you  may  con- 
tradict the  topic  with  an  opposing  point  of  view. 
Attempt  to  build  a  logical,  reasonable  opinion 
paper.     Include  an  introductory  and  concluding 
paragraph     and  two  or  three  paragraphs  in  the  body 
of  the  paper.     The  supporting  paragraphs  aay  be 
developed  by  comparison/contrast,  definition, 
illustration,  or  any  other  form  which  will  produce 
a  unified  essay  on  Chia  topic.     (Jefferson  County, 
Colorado,   Writing  Sample  Report ,  1976.) 
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PROMPT  UQa 


'•Write  a  letter   to  your  principal.     Explain   to  him 
or  ner  what  you  would   like  to  5ee  changed   in  your 
school.''   (C,   Keech,  example  of  proble^Tj) 


PROHPT  ^Sb 

"There  are  probably  several   things  you  would  like 
to  see  changed   In  your  school.     CHDDSE  ONE  OF 
THESE  and  write  a  letter  to  your  principal, 
describing  or  explain ing  the  problem  and  suggest- 
ing what  he  or  sne  might  do  to  change  the  sltua- 
tlon.«   (Variation  of  HAEP  ••letter  to   the  princi- 
pal"  Writing  Report ,   05-W-03f    1977  by  C.  Keech.) 

PROrtPT  48c 


There  is  probably  at  least  one  thing  about  your 
school   that  you  would  like  to  see  changed.     Write  " 
a  letter  to  your  principal  and  tell   hia  about  the 
problem.     Be  sure  to  describe  the  situation  as  it 
Is  now  and  tell  why  it  bothers  you,  as  well  as  how 
you  would  like   to  change  it.     Variation  of  HAEP 
"letter  to  the  principal".  Writing  Report^  05-W- 
03.    1977  by  C.  Keech.) 

PROrtPT  49 


The  following  article  appeared   in  the  local  news- 
paper : 

KDOL  Begins  Survey: 

Wny  do  we  remember  certain   people  from  our  past 
and   forget  others?     Television  station  KDOL  is 
preparing  a  special  TV  show  on  personalities  peo- 
ple remember  from  their  elementary  school  years--a 
classmate,  a  member  of  the  school  staff,  a  family 
member,  or  a  neighborhood   friend.     Send  your 
description  to: 


Program  Producer 
KDOL 

Oakland  ,   CA  94606 

Respond  to   the  article  by  choosing  someone  ,who 
aade  an  unusually  strong  impression  on  you.  Write 
about  thit  person,  explaining  why  you  remember  him 
or  her.     Do  your  best  work,     (Miles  Myers,  Oakland 
Schools  Assessment.) 
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PROMPT  50 


"Write  a  friendly  letter  to  a  pen 
country,  telling  him  or  har  about 
school  . "  ( Comaonl y  used  to  pi  c  . ) 

PROMPT  51a 


pal  in  another 
your   home  or 


for  so-ae  rea- 
Hrite  abojt 
your  reader 
this  nemory 


Think  of  an  occasion  when  you  experienced  some- 
thing  for  the  FIRST  TIME.     It  nay  have  been  some- 
thing you  later  came  to  do  more  easily  or  some- 
thing you  now  take  for  granted;  but, 
son,  your  first  time  was  me.norable. 
this  "first  time"  experience  so  that 
can  understand  your  feelings  and  why 
has  stayed  with  you,     (C.  Keech.) 

PROMPT  51b 

Think  of  an  occasion  when  you  experienced  some- 
thing  for  the  FIRST  TIME.     It  may  have  been  some- 
thing you  later  came  to  do  more  easily  or  some- 
thing you  now  take  for  granted;   but,  for  some 
son,  your   first  time  was  memorable.  Imagine 
you  are  writing  to   someone  who  has  Just  had  a 
similar  experience,   OR  to  someone  who  is  about  to 
have  such  an  experience.     Your  writing  might  help 
prepare  your  reader  for  the  experience 
might  help  your  reader  understand  that 
pie  have  gone  through  the  same  kind  of 

(C.  Keech), 


r  ea- 

that 


or  it 
other  peo- 
tning  . 
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Chapter  2 


EFFECTS   OF  VARIATION  IN 
UPOIs  HOLISTIC  SCORE 

Catharine  Keech,  Don 
Charles 


A  WRITING  TEST  PROMPT 
AND   OTHER  FACTORS 

Leu ,    Sandra  Murphy , 
Kinzer 


Part    I:        Effects   of  Two  Versions  of  a  Writing 

Prompt   Upon  Holistic   Score   and  Writing 
Processes 

Don   Leu,    Catharine  Keech,    Sandra  Murphy, 
and   Charles  Kinzer 


Part  II:  Comparison  and  Analysis  of  Rater  Responses 
to  the  Anchor  Papers  in  the  Writing  Prompt 
Variation  Study 

Catharine  Keech   and   Mary  Ellen  McNelly 
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Chapt  er  2 : 


Part  I 


i 


EFFECTS   OF  TWO   VERSIONS    OF  A  WRITING   PROMPT  UPON 
HOLISTIC   SCORE  AND   WRITING  PROCESSES* 


Don  Leu,    Catharine   Keech,    Sandra  Murphy, 
and   Charles  Kinzer 


OVERVIEW  OF  THE  STUDY 


To  investigate  effects  of  variation  in  the  specif- 
ication of  audiencn  on  a  timed  writing  test«    11M  high 
school  students,  grades   10-12,   from  four  classes  of  one 
teacher,  were  randonly  assigned  one  of  two  versions  of 
a  thirty-ninute  test  asking  them  to  write  about  a  par- 
ticular personal  experience:  an  occasion  when  the 
writer  experienced  something  for  the   first  time.  The 
subject  and  the  time  limit  are  typical  of  writing  tests 
used  in  assessments  of  writing  ability  by  many  local 
school  districts.     The  essays  were  ranked  ordered  by 
quality  following  holistic  rating  procedures. 

Data  about  the  subjects  were  collected  to  provide 
insight  into  additional   factors  other  than  the  task 
variations  which  might  also  affect  the  quality  of  per- 
formance as  measured  by  the  holistic  score.  Estimates 
of  verbal  skill  and  attitude  were  obtained  by  having 
the  teacher  rate  subjects  individually  on  the  basis  of 
classroom  behavior  and  performance.     Data  on  sex  and 
academic  proficiency  level  were  also  recorded.     A  pro- 
ficiency level  was  assigned  on  the  basis  of  the  sub* 
Jects'   grade  level  and  academic  track  placement., 

Finally,  the  day  after  writing  in  response  to  one 
of  the  two  versions  of  the  "first  time"  prompt,  sub- 
jects in  the  study  completed  a  questionnaire  designed 
to  elicit  information  about  their  attitudes  toward  the 
prompt,   their  recollections  of  the  writing  process  dur- 
ing the  timed  writing  test,  and  their  descriptions  of 
their  other  wr 1 t 1 ng  beha v lo r s  and  attitudes,  both  in 
and  out  of  school. 


Associations  between  holistic  score,  ^erslon  of 
prompt,  teacher  rating  of  verbal  akill,  teacher  rating 
of  attitude,  sex  of  the  writer,  proficiency,  and 
responses  to  questionnaire  items  were  examined.  Two 
questions  were  of  particular  Interest: 


•Thrs  report  is  based  upon  research  supported  by  the 
National  Institute  of  Education  under  Grant  No.  HIE  G- 
80-003^  to  the  Bay  Area  Writing  Project,  University  of 
California,   Berkeley,   Leo  Ruth,   Project  Director. 
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1)       Wnlch  variables  are  associated  with  hign 
holis  tic  scores? 


2)       Which  variables  are  associated  with  differ- 
encesintheprorapt? 

r 

The  da-ta  yielded  interesting  Information  in  response  to 
both  questions,   suggesting  differences  in  prompt 
affected  specific  aspects  of  the  writing  process 
although  the  mean  quality  of  performance  as  measured  by 
holistic   scores  was   similar   for  both  versions.  Unex- 
pected patterns  of  response  on  the  questionnaire,  for 
example,   indicated  differences  in  the  planning  process 
with  an   increase  in  planning  time  associated  with  the 
minimal  audience  Specification. 

Although  one  task  constraint,  degree  of  audience 
specification,   was  manipulated  experimentally,  the 
st-udy  as  a  whole  should  be  viewed  as  descriptive  and 
ex'planatory  rather  than  as  an  exactingly  controlled 
Experiment.     Viewed   in  this  light  the  study  offers  a 
number  of  provocative  leads  for  needed  investigations. 


PROCEDURES   OF  THE  STUDY 


Subjects 

The  subjects  of  the  study  were  114  students  at  Los 
Gatos  High  Schoolt   located   in  Los  Gatos,  an  upper-* 
middle  class,  suburban  community  in  Northern  Califor- 
nia.    The  subjects  were  students  In  four  classes  taught 
by  one  teacher  whose  program  differs  somewhat  from  the 
traditional  English  program  generally  followed  at  the 
school.     The   traditional  program  emphasizes  the  teach- 
ing of  graumar  and  Instruction  In  basic  skills  with  a 
concentration  on  the  forms  of  exposition  In  upper  grade 
level  courses.     The  teacher  in  this  study,  a  teacher- 
consultant  with  the  Bay  Area  Writing  Project,  Mary 
Ellen  HcMelly,  emphasizes  writing  from  personal  experi- 
ence, providing  more  opportunities  for  expressive  writ- 
ing than  might  be  found  in   other  classes  at  the  school. 

The  students  in   the  school^ are  tracked,  according 
to  ability  level,  into  accelerated  and  non-accelerated 
English  programs.     Tracking  decisions  are  based  largely 
on  two  factors:  teacher  recommendations  and  performance 
on  a  test  constructed  by  the  English  department.  Tne 
departmental  test  consists  of  both  an  objective  exami- 
nation and  a  writing  sample.     The  objective  examination 
covers  usage,  mechanics,  vocabulary,  and  spelling.  The 
writing  sample  is  evaluated  on  the  basis  of  both  form 
and  content. 

The   114  students  who  make  up  the  population  of 
this  study  came  from  one  section  of  advanced  composi- 
tion juniors  and  seniors  (N  »  22),  two  sections  of 
college-track  English  literature   Juniors  (N  «  65),  and 
one  section  of  general  English,   average-ability  sopho- 
mores (M  «  27).     The  87  Juniors  and  seniors  in  advanced 
composition  and  English  literature  were  in  the 
accelerated  program  at  the  school.     The  27  sophomores 
in  general  English  were  in  the  non-accelera t ed  program. 
There  were  66  female  subjects  and  48  males.     Tne  number 
and  percentage  of  students  in  each  category  can  be  seen 
in  Table  1.* 

Determination  of  Verbal  Skill  and  Attitude  Ratings 

The  teacher  provided   subjective  impression  ratings 
of  students*   verbal  skills  and  attitudes  toward  achiev- 
ing in   school,  two  factors  which  might  be  expected  to 
bi*  associated  with  performance  on  a  writing  test.  At 
the  end  of  the"  school  year ,  when   the  testing  took  place 


•All  tables  and  figures  appear  at  the  end  of  this  re- 
port. 
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(May  1931),  the  teacher  could  be  expected  to  have 
formed  a  clear  impression  of  her  students'  capabili- 
ties, especially  since  her  particular  teaching  style 
was  informal,  personal  and   Interactive.     A  teacher  with 
20  years  of  experience,  she  could  also  be  expected  to 
have  established  intuitive  "norms"  or  expectations  for 
performance  at  a  given  age  level.     The  teacher  was 
asked  to  draw  on  her  total   impression  of  a  student 
gained  from  -hat  she  had  observed  of  his  or  her  ability 
to  comprehend   literature  and  to  express  himself  or  her- 
self orally  and  in  written  English  on  a  variety  of 
occasions,  both  formal  and  informal.     She  was  then 
asked   to  assign  each  student  a  rating  from  1-U  on  a 
factor  called  general  verbal  ability:   1  «  well  below 
grade  level;  2  «  grade  level  or  below;   3  *  grade  level 
or  above;  and  U  =  well  above  grade  level.     She  was  also 
asked  to  assign  each  student  to  one  of  four  categories 
based  on  her  evaluation  of  the  student's  jttltude 
toward  achieving  in  school:   1  *  extremely  negative,  2  = 
mildly  negative,   3  *  «lldly  positive,   U  «  extremely 
positive.     A  "positive  attitude"  would  indicate  that 
the  student  appeared  motivated  to  perform  well  in 
school  and  seemed  to  be  working  to  his  or  her  best 
ability . 

Timed  Writing  Test  Versions 

This  writing  test  occasion  required  students  to 
compose  a  response  to  one  of  the  following  two  ver- 
sions*  of  a  personal  narrative  prompt  within  thirty 
minutes: 

PROMPT  VERSION  A: 


Think  of  an  occasion  when  you  experienced 
something  for  the  FIRST  TIME.     It  may  have 
been  something  you  later  came  to  do  more 
easily  or  something  you  now  take  for  granted; 
but,  for  some  reason,  your  first  time  was 
memorable.     Write  about  this  "first  time" 
experience  so  that  your  reader  can  understand 
your  feelings  and  why  this  memory  has  stayed 
with  you. 

You  may  write  an  essay,  a  short  story,  a 

•fnTTIFlable  sets  ofTTTItr uctlon  In  the  two  prompts 
are  presented  here  enclosed  In  boxes  for  purposes  of 
comparison  In  this  report.  The  students .  however ,  re- 
ceived the  test  Instrument  In  the  format  appearing  in 
Appendix  A,  with  only  one  version  of  the  prompt  appear- 
ing m  the  writing  test. 
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letter,  or  a  journal  entry  as  a  way  of  re- 
telling your  experience. 


PROMPT  VERSION  B: 


Think  of  an  occasion  when  you  experienced 
sonething  for   the  FIRST  TIME.     It  may  have 
been  something  you  later  came  to  do  more 
easily  or  something  you  now  take  for  granted; 
but,  for  some  reason,  your  first  time  was 
memorable.     Imagine  that  you  are  writing  to 
someone  who  has  ^ust  had  a  similar  ^1^^^^' 
ence,  OR  to  someone  who  is  about  to  have,  such 
an  experience.     Your  writing  might  help 
prepare  your  reader  for  the  experience,  or  it 
night  help  your  reader  understand  that  other 
people  have  gone  through  the  same  kind  of 
thing . 

You  may  write  an  essay,  a  short  story,  a 
letter,  or  a  journal  entry  as  a  way  of  re- 
telling your  experience. 

The  pers<>nal  narrative  topic,   "Tell  about  a  first  time 
expeJ  ende-.   wa.  chosen  as  being  typical  of  P';o»f  J 
used  in  high  school  writing  assessments  J ^Jj^" 

c=illy.     Such  topics  are  often  used  because  they  assure 
that  ;i«ost  eve^y  student  will  find  material  for  wrlt- 
ing  in  personal  Jxperlenoe  and  be  able  to  begin  compos- 
inl  quickly.     These  prompts  aim  to  elicit  "»Pl" 
general  fluency  In  written  .English ,  drawing  on  "Jttrlal 
flSklTts  may  have  already  told  orally.     Because  this 
prompt,  especially  Version  A.  does  not  require  special 
-rltlni  skills  such  as  argument  or  expos  tlon.  It  can 
Je  used  across  several  grade  or  ability  levels  and  In 
different  courses. 

The  major  difference  between  the  two  versions  of 
the  prompt  Is  the  more  explicit  specification  of  the 
.S5ienc°'and  rhetorical  purpose  , 
v.rslons  however ,  ultimately  .re  directed  ^o  an  Impli- 
cit audience  of  test  readers  because  the  writing  occurs 

the  context  of  .  testing  .Ituatlon.     J"  Version  B 
the  instructions  specify  an  1«-Si"»;;y  ■"Ji!"^?' 
one-  who  has  shared  or  will  share  ^^f /^[J^  .^^'^^  y^^. 
experience  the  student  chooses  to  write  about.     J"  Jer- 
sJon  A  specification  of  the  audience  Is  less  restricted 
and  refers  simply  to  "your  reader." 

This  study  of  effects  of  variation  In  prompt 
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wording  builds  on  a  study  reported  In  The  Write  Ocea- 
slon  (Woodtforth  and  Keeeh,   1930)  which  set  out  to  test 
a~eom.iion  assuaptlon  by  test-makers  that  specifying  the 
a-jdlence  and  rhetorical  purpose  enhances  the  quality  of 
student  per  foriaan  ce .     That  study  failed  to  find  a 
difference  In  nean  holistic  scores  between  the  two  ver- 
sions, but  hypothesized  several  explanations  for  this 
result.     The  present  study  probes  further  for  differ- 
ences that  may  not  be  reflected  In  holistic  scores, 
through  the  administration  of  a  post-test  question- 
naire.'   One  Important  reason  for  continuing  to  examine 
variations  in  the  wording  of  prompts  even  when  they  do 
not  appear  to  significantly  affect  holistic  scores  is 
to  determine  how  student  writers  construe  the  writing 
task.     The  research  goal  is  to  identify  possible  con- 
straints affecting  the  process  of  interpretation  of 
task  demands  by  student-writers  and  evaluators  of  the 
papers  elicited.     This  knowledge  will  contribute  to  the 
development  of  a  model  of  the  whole  writing  assessment 
eplsod  e . 

Administration  of  the  Test  and  the  Holistic  Rating 

The  two  versions  of  the  prompt  were  distributed 
randomly  by  the  regular  teacher  in  the  four  classes  (in 
May  1981)   so  that  equal  numbers  of  students  in  each 
group  wrote  on  each  version.     The  teacher  distributed 
the  tests  and  read  through  the  instructions  (See  Appen- 
dix A  for  complete  instructions  to  students)  with  stu- 
dents, pausing  for  students  to. write  their  individual 
code  numbers.     Test  conditions  were  consistent  for  all 
four  classes.     Ten  students  were  taken  out  of  the 
classes  to  do  the  writing  test  under  special  conditions 
'requiring  oral  composing,  under  the  supervision  of  the 
investigator  ,  Catharine' Keech.* 

T  Each  paper  received  three  sets  of  holistic 
scores.*  First,  the  papers  were  randomly  mixed  across 
all  classes  and  grade  levels,  then  distributed  to  peer 
raters,  who  had  received  special  training  and  who  were 
the  first  nolistic  raters  of  the  essays.     A  second  set 
of  scores  was  provided  later  when  the  researcher  Catha- 
rine Keeeh  led  a  group  of  student-teachers  at  the 
University  of  California,  Berkeley  in  rescorlng  the 


•The  ten  oral  composing  protocols  thus  elicited  have 
not  been  analyred.  but  the  ten  papers  produced  are  in- 
cluded in  the  reporting  of  all  results  below,  having 
been  scored  as  part  of  the  larger  sample  of  papers. 
•Tne  three  rating  sessions  are  described  in  Part  II  of 
the  report  wnere  results  of  the  training  sessions  are 
discussed  and  comparisons  of  the  three  sets  of  scores 
are  reported. 
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papers,  using  training  techniques  similar  ^^^/J^^/J' 
LI  usid  -ith  the  oriain.l  student  raters.     A  third  set 
of  scores  was  provided  shortly  after-ards  by  four 
-expert"  holistic  raters  -ho  -ere  teacher/consultants 
for   the  Bay  Area  Writing  Project.     This  "ird  set  of 
scores,  assigned  by  the  expert  raters  in  a  blind  rank- 
ing, provides  the  scores  for  the  analysis  reported 
below . 

In  general,   the  training  of  the  expert  holistic 
raters  follo-ed   q^on ventional  procedures.     Tne  four 
experts  ranked  anchor  papers  selected  by  the  researcher 
Catharine  Keech .  acting  as  trainer.     Scores  -ere 
assianed  by  each  rater  on  a  scale  of  l-M,  with  i 
assUSed  to  the  poorest  papers  and  4  to  the  best  papers 
The  anchor   papers  sho-ing  the  quality  °f  -""^  J'"*  '^^^ 
each  scoring  level  are   included   in  Appendix  B.  Since 
eJch  paper  -as  scored  by  t-o  raters,   the  total  score 
"raSy  paper  could  range  from  a  mini-am  of  2  ta  .  max- 
imum  of  8. 

Th-  expert  raters  quickly  reached  consensus  on  the 
criteri;  to  be  applied   in  rating  the  "JJ^J, 
They  achieved  a  significant  level  of  ^^^^'''J  Jl^J  * 
Pearson  product  moment  correlation  between  the  first 
and  the  second   scorer  was  r  s   .71   (P  <  .01). 
Discrepancies  (t-o  scores  given  to  the  .  ^^^^ 

are  more  than  one  point  apart)   -ere  resolved  1"  the 
usual  -ay  in  a  third  reading  by  the  researcher  acting 
as  the  table  leader. 

Administration  of  the  Questionnaire 

Dittoed  copies  of  a  typed  questionnaire  consisting 
of  25  items  were  distributed  to  the  students  during 
their  regularly  scheduled  class  periods  on  the  day 
after  the  -riting  test.     Complete  questionnaire  data 
were  collected  on  99  of  the   11U  students. 

The  questionnaire  -as  divided  into  three  sections. 
In  the  first  section,   students  -ere  asked  to  ["^three 
iersions  of  the  prompt.     Two  of  the  versions  had  been 
assigned  for  the  writing  test  (Prompt  Versions  A  and  B 
\loA).     Tne  third  version  -as  ne-  to  the  entire  group: 

PROMPT  VERSION  C 

There  is  a  popular  book  called  Mjr  First  Time, 
edited  by  Bennett  Cerf.  which  is  a  collection 
of  pieces  by  famous  people  -ho  are  describing 
experiences  they  remember  -hen  they  -ere 
doing  something  for  the  first  time.  Thsy 
tell  about  first  kisses,   first  time  on  the 
stage,  first  crime,   first  time  they  sa- 
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someone  famous  they  would  later  work  with, 
first  political  caupalgnt   first  time  they 
were  afraid,  etc.     Some  of  their  experiences 
are  unusual,  some  are  common  but  well  told.^ 

I  would  like  to  collect  a  book  of  first  Time 
experiences  of  high  school  students.     I  have 
asked  a  number  of  teachers  to  all,ow  their 
students  to  write  about  their  First  Tljme^ 
during  a   30  minute  writing  test,   from  which  I 
will  select  promising  pieces.     Later  I  will 
contact  the  authors  of  the  selected  pieces 
and  give  them  an  opportutiity  to  revise  and 
polish  before  publication.     Chosen  pieces 
will  be  copyrighted  and  the  authors  will 
receive  $50  for  their  contribution.     No  piece 
will  be  used  without  the  author ' s  permission. 

You  may  write  an  essay,  a  short  story,  a 
letter,   or  a  Journal  entry  as  a  way  of  re- 
telling your  experience. 


Students  were  asked  to  identify  which  version  they 
actually^had  been  given,  and  also  which  version  they 
might  have  preferred  to  write  on  if  they  had  been  given 
these  three  choices.     Space  was  provided  for  comments 
regarding  preference. 

In  the  second  and  third  sections  of  the  question- 
naire, students  responded  to  .statements  (e.g.  "I  con- 
sider myself  a  good  writer.")  on  a  four-point  Likert 
scale:  strongly  agree,  agree,  disagree,  strongly 
disagree.     A  copy  of  the  complete  questionnaire  may  be 
found  in  Appendix  B.  * 

Choice  of  Statistical  Tests 

In  reporting  agreement  between  readers  during 
holistic  scoring.  Pearson's  product  moment  correlation 
(r)   Is  used  as  a  measure  of  association  comparing 
scores  given  by  the  first  and  second  readers  on  every 
essay,   following  common  practice  in  holistic  assess- 
aen  t . 

With  the  exception  of  that  special  use  of 
Pearson's  r.  the  statistical  measures  used  in  this 
report  are  non-parametric,  specifically  tau  and  chi 
square  (X^).     Non-parametric,  measures  are  appropriate 
rH^Tesearch  where  the  number  of  cases  cnanges  across 
comparisons  and  where  those  comparisons  were  not 
planned  as  a  part  of  the  design  of  the  study,  but  are 
more  in  the  nature  of  tentative  seekings  of  possible 
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relationships.     Because  the  comparisons  are  unplanned, 
in  this  sense,  rather  than  part  of  a  tightly  controlled 
hypothesis  testing,  it  is  laportant  to  use  the  most 
conservative  possible  estimates  of  association  of  vari- 
ables.    Non-parametric  statistics  serve  this  purpose, 
aaklng  it  more  difficult  to  find  statistical  signifi- 
cance.    Tau  is  a  measu.e  of  association,   like  r. 
expressiTTT'a  relationship  bet-een  an  independent  and  a 
dependent  variable  in   terms  of  a  range  of  values  from 

00  to  1.00.  with  lower  values  representing  a  weak 
d.aree  of  association  and  higher  values  representing  a 
st?ong  association.     It  is   appropriate  -hen  both  vari- 
ables provide  a  lirear  set  of  values  from  lower  to 
higher  numbet-s. 

Cni   Square   (X^)   is  not  a  measure  of  association  in 
the  same  sense;  rather  it  is  a  simple  yes-no  test  for 
the  non-independence  of  two  discreet,  categorical 
independent  var iables  .  each  having  only  t-o  values.  * 
and  -       The  value  of        must  be  greater  «hen  the  number 
of  cases   is  smaller,   in  order  to  demonstrate  the  non- 
independence  (or  association)  of  t-o  ^^^^Jf 
tistically  significant.     In  reporting  a  ^g^jjionship 
b-tv^een  t-o  such  variables,  -e  give  the  X     value  in 
pirentneses.   together  -ith  the  M  of  cases,  to  demon- 
strate statistical  significance  at  the  p  x  .05 
(These  numbers  do  not  represent  degrees  of  association, 
as  do  the  puflbers  reported  as  values  for  tau) 

Responses  to  the  questionnaire  items  can  be  most 
understandably  reported  in  relation  to  other  variables 
if  the  Liket   four-point  scale  is  reduced  to  a  t-o-part 
categorical  variable,  -ith  positive  responses  to  an 
item  set  against  negative  responses.     Since  many  of  the 
variables  being  related  to   the  questionnaire  items  are 
themselves  categorical,  chi  square  (X  )   is  the 
appropriate  test  to  determine  whether  student  attitudes 
as  revealed  in  the  questionnaire  are  in  any  -ay  associ- 
ated -ith  other   independent  variables  such  as  sex.  pro^ 
ficiency   (determined  by  placement  in  cl as ses )  .  mode  of 
-riting   (collapsed   into  t-o  ca te gor i es .  nar r a t i ve  and 
non-narrative),  etc.     It  also  appropriately  tests  rela- 
tionships bet-een  items  on  the  questionnaire. 
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GENERAL  RESULTS 


Tne  following  sections  provide  descriptive  data 
about  the  characteristics  of  the  population  tested  and 
the  relationships  of  these  data  to  holistic  scores. 
Interaction  between  variables  and  their  relation  to 
holistic  scores  are  also  reported  here.     And  finally, 
the  descriptive  analysis  of  the  questionnaire  data  is 
pro  V  ided . 

Verbal  Skill  ,   Attitude  ♦  and  Hoi ist ic  Score 

The  population  distribution  for  verbal  skill  rat- 
ing, attitude  rating,   and  perfornance  on  the  timed 
writing  test  as  measured  by  the  holistic  scoring  are 
presented  in  Tables  2,   3,   and  4.     The  distribution  of 
ratings  is  positively  skewed  on  all^three  variables, 
indicating  that  most  of  the  students  possess  well- 
developed  skills  and  favorable  attitudes  toward  their 
class  and  work.     This  was  expected  since  three  of  the 
four  class  sections  are  accelerated  courses  for  highly 
able  and  ambitious  students. 

Significant  associations  were  found  between  all 
three  variables.     Tne  correlation  between  the  teacher's 
estimate  of  verbal  skill  and  attitude  was  significant 
and  positive   (Tau     r   .583,  p  <   .05).     Significant  posi- 
tive correlations  were  also  found  between  verbal  skill 
and  performance  on  the  timed  writing  test  as  measured 
by  holistic  score  (Tau     x   .33.   P  <   -05)  and  between 
attitude  and  holistic  score  (Tau     «   .35.   P  <   .05).  An 
increase  in  holistic  score,   among  these  students,  is 
associated  with  an  increase  in  attitude  rating  and  an 
increase  in  verbal  skill  rating;  students  with  high 
holistic  scores  tend  also  to  have  high  attitude  and 
verbal   skill  ratings.     Moreover,  the  association  of 
attitude  rating  with  holistic  score  is  slightly 
stronger  than  the  association  of  verbal  skill  and 
holistic  score. 

Sex 

Mo  significant  relationships  were  found  between 
sex  and  holistic  score,  verbal  skill   rating,  or  atti- 
tude rating.     Both  males  and  females  performed  at  the 
same  level  of  proficiency  on  the  timed  writing  test. 
Both  sexes  also  had  similar  verbal  skill  ratings  and 
attitude  ratings. 

Proficiency  Level 

In  this  investigation,  proficiency  level  was 
defined  by  each  student's  course  placement;   that  is, 
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•  ^Pniors  in  tne  advanced  composition 

tha  Tuniors  and  seniwra  xu  wis-  -  c«,»i<«h 
^?asa  and  the  Juniors  in  tne  two  sections  of  English 
fit^hatuJe  i're  in   tna  accelerated   traCK  and  ware. 

errrore!  :;n:.dered  to         -^^^^^^^ J^J^^  ^f,  ' 
«rs.     Tne  sophonoras  in   the  seneral  English  class  --r 

I  ^^ir  vnitney  U  T-st.     Juniors  and  Seniors  In  tns 
corie^atirtJac.  Perror.e.  sUnlfloantly  ..tt.r  on  e 

jirLr-i":^J:"^';raerru°r;'o!ro!'r:  "i^.-p  <  • 

05)       ^na  »ean  ran.  of  th.  proficient 

:Jrrns"f'pjrflot:n?';"-l.ss  prof>elant  -rlt.rs.  .o»l 
holistic  scores  can  be  seen  in  Table  5. 

mere  was  no  relationship  between  the  sex  of  the 
student   and  proficiency   level*   ^  f  ^2  =  °- %1» al es 
05).  meaning  that  proportions  of  males 

r.u;;"U-.l^;r;:J;;9;;;"r^•:;^^L■^•,v.r.^■"• 

^.ra  considered  above  grade   level   in  verbal  skill  co^- 

i-r.  oniv  ^7  Oi   (N  =  10)   of  tne  less  proficient  writ- 
pared  to  °nly  37.0%  ■>  ^  ^^.^g)   ^^^e  con- 

rid;ref  b       Jei^-fearhir   to  have  a   positive  attitude 
toward  achieving  in  school  compared  to  only  33.3*  (N.9) 
of  the  Tess  proficient  writers.  ^     ^^"^j; ! ,  ' 

sL^s  to  be  importantly  related  to   verbal  slcill  and 
attitude  in  addition  to  total  holistic  score. 

Effect  of  Prompt  on  Score  and  Selection  of  Mods 

Tn-  two   versions  of  the  prompt  were  distributed 

?£risrrfn:^/;:;rai"s.t!i"ar;uiL^2  rrn»,n:.- 

5>its       Likewise     scores  on   the  two  prompts  were  com- 
prrabie.     Th^re  :as  no  association  evident  between  the 

TvT^TTT'I^r^lTTTriiTF^  JF^Tducted     after     collapsing  the 

fan-  i^ii/nnn':n%'rf?arsii!ra;!/-ruUs: 

cl"  atuLi,  c,te,  =  rl.s  Into  t«  (n.^atlv,  and 
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assigned  proiipt  and   the  holistic  perfornance  score 
anong  the   114  students  who  co.*npleted   the  writing  test: 
the  mean  ranks  of  total  holistic  scores  were  no^ 
f  icantly  different  for   the   two  versions   (^^3^^^  • 
=54.97,   X  =60.03.   P  >   .05).     Tne  mean  score^for 

pro:npt  A  /^S   i*09t   the  mean   for  prompt  B  was  6,23. 

An  examination  of  the  students'    papers  showed  that 
different  genre  were  used  in  responding  to   the  two  ver- 
sions of  the  prompt   (e.g.,   narrative,   letter,  journal, 
essay).     An  analysis  was  then   per  formed  to   find  whether 
tne   prompt   itself  had  any  effect  on   selection  of  the 
mode  of  writing   (see  Table  6). 

The  narrative  node  was  the  dominant  choice  of 
writers  on  both  tasks.     Even   so,   some  writers  made 
other  choices.     Although  76.3*  of  students  wrote  a  nar- 
rative,  18. MX  wrote  a  letter,   4. MX  wrote  a  journal 
entry,   and   0.9X  wrote   an  essay  in   responding  to  the 
prompts.     In   all,   a  total  of  27  students   (23.7*)  chose 
alternate,   or  n 0 n -na r r a t i v e  modes  of  writing. 

Tne  effects   of  the  prompts  are  clearly  associated 
with  the'student  •  s  choice  of  mode.     Although  equal 
numbers  of  students  wrote  on  each  version,  more  Version 
B  students  produced  alternate  modes   of  writing.  Stu- 
dents who  wrote  on  Version  A  produced  7  alternative 
mode  papers   (5  letters  and  2   journal   entries)  while 
students  who  wrote  on   Version  B  produced  20  alternative 
mode  papers   (16  letters,   3  journal  entries,   and  1 
essay).     Almost   three   times  as  many  non-narrative  mode 
papers  were  produced  by  students  writing  on  Version  B« 
Most  of  the  alternate  node  papers  (96. 3X  or  26  of  27) 
exhibited  personal  forms  of  writing  such  as  letters  or 
journal  entries.     Most  of  these  more  personal  forms 
occurred  in   response   to  Version  B,   the  version  which 
more  sharply  identified  an   audience  for  the  writer. 

There  also   was  a   significant  relationship  between 
sex   and  tne  mode  of  writing   selected  by  students. 
(  =  8.65,   P  <   .05).     A  higher   frequency  of 

females   (females  =  33.3X,  males  =   10. 6X)   chose  a  non- 
narrative   form.      In  addition,   females  varied  more  in 
their  choice  of  non-narrative  modes.     Females  wrote  15 
letters,   5  journal  entries,   and   1   essay.     The  five 
males   wno  chose   a  no n -nar r a t i v e  mode  all  wrote  letters 
(See  Table  7). 

Howev  er ,   there  was  no   significant  relationship 
between  tne  mode  that   was  selected  and  eitner  verbal 
3i<ill   rating   (X^   =  3.42,   p  >    .05)   or  attitude  rating 
(X     =7.83,   p  >    .05).     There  was  also  no  significant 
difference   in  the  mean  ranking  of  holistic  scores  for 
narrative  and  no n-na r r at i v fe  mode  papers  (X  narrative  = 
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55. MO,   X  non-nirratlve  =  61.06,  p  >   .05).     The  nean 
total  holistic  score  for  narratives  was  6.13,  while  fo 
non-narratives  it  was  6. MO.     Finally,   there  was  no 
•  ppa-rent  interaction  between  aode  selection  (narrative 
and  non-narrative  modes)   and  proficiency  when  total 
holii'tle  score  was  used  as  the  dependent  measure. 
Table  8  indicates  that  both  proficient  and  less  profi- 
cient students  perforaed   siailarly  whether  they  util- 
ized  a  narrative  or  a  non-narrative  aode. 


4  «> 


227 


QUESTIONNAIRE   DATA  AND  COMMENTARY 


The   following  descriptive  data  are  based  on  the 
responses  obtained  on  99  questionnaires.     Each  data 
report  Is  followed  by  a  brief  oonaentary. 

Memory  For  The  Assigned  Prompt .     When  asked  to 
indicate  which  version  of  the  prompt  they  had  been 
assigned   for  the   test.    17  students  ( 1 1 . 9» )  responded 
incorrectly;  I.e.  they  thought  they  had  written  on  a 
particular  version  when  they  actually  had  not.     Most  of 
the  students  who  responded   Incorrectly  (Ns13)  actually 
wrote  on   Version  B;  eleven  who  wrote  on  Version  B 
remembered   It  as  Version  A;  two  who  wrote  on   Version  B 
remembered   it  as  Version  C.     Only  four   students  who 
responded   Incorrectly  actually  wrote  on  Version  A;  all 
remembered   It  as  Version  B. 

Commentary:  The   fact  that  about    1  In   6  students  failed 
to   Identify  the   prompt   that  they  wrote  on  seems  likely 
to  be  due   to   the   similar  wording  In   Versions  A  and  B. 
This  Interpretation  is  supported  by  the  observation 
that  most  studfcnts  (Nr15)  mistakenly  Identified  the 
prompt  they  wrote  on  as  either  A  or   B.     Only  two  stu- 
dents mistakenly  Identified  Version  C  as  the  topic  they 
wrote  on 


The   fact  that  more  Version  B  students  thought  they 
wrote  on   Version  A,   rather  than  vice  versa,  may  be  due 
to   the  order  of  the  versions.     Some  of  Version  B  stu- 
dents may  have  selected  Version   A  after  reading  only 
the   first  several  llnea.     The   first  two  sentences  of 
Version   A,    It  should  be  noted,  are  Identical   to  ttie 
first  two   sentences  of  Version  B.     The   final  sentence 
also  is  Identical   In  both  versions. 

Task  Preference .     A  majority  of  the  respondents 
(51 . 1t)~Ind leattd   they  had  no  preference  In  regard  to 
the'three  versions  of  the  prompt.     Of  those  who  had  a 
preference.   39J  preferred   A,    7»  preferred  B.   and  5n 
preferred  C. 

Commentary:   One  observer   suggests  that  the  greater 
preference  for   Version   C,   which  provides  an  elaborate 
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reality  simulation,  may  simply  be  a  result  of  the  fact 
that  this  was  the  version  not  written  on.  This 
hypothesis  could  be  easily  tested  by  allowing  students 
to  choose  which  of  the  three  versions  they  wanted  to 
respond  to  during  an   actual  writing  test.     It  is  also 
possible  that  students  preferred  Version  C  because  it 
sounded  least  like  a  writing  test  prompt.     In  other 
words,   they  thought  they  would  prefer  writing  for  the 
situation  represented  by  Version  C  than  for   the  test 
versions ,  A  and  B. 

Hecollections  About  The  Planning  Process  .*  A 
majority  of  the  students  T65.7»)   indicated   they  had 
enough  time  to  complete  the   task  but  more  than  a  third 
of  the   students  (3U.3*)  reported  they  did   not  have 
enough  time  to  write.     Most  students  (70. 7»)  indicated 
they  had  spent  a  lot  of  time  (at  least  5  or   10  minutes) 
thinking  about  what  they  wanted  to   write  before  they 
started.     This  is  an  unusual   finding  for  impromptu 
timed   tests   (cf.   Pianko,    1979;  Emig.    1971)  but  confirms 
the  observations  of  the  teacher  during  test  administra- 
tion.•  About  a  third  (3"*. 3*)  of  the  students  reported 
that  they  felt  rushed  when  they  started  writing  because 
they  spent  "so  much"   time  planning.     Most  atudents  did 
not  re-read   their  papers.     About  half  (50.5%)  indicated 
they  did  not  have  time  to  re-read,  while  an  additional 
third  30. 3X  indicated  that  although  they  finished 
before  the  time  was  up,  they  did   not  re-read  Most 
students  (72. 7S)   indicated   they  would  have  written  very 
differently  about  the  prompt   if  they  could   have  taken 
the  assignment  home  to  complete  it.  or  if  they  had  had 
several  days  In  class  to  work  on   it.     A  similar  number 
of  students  (71.  1»)   reported  they  could  have  done  a 
better  Job  on  the  assignment  if  they  had  been  given  a 
full   hour  for  writing.     Only  20.2%  of  the  students, 
however,  reported  that  they  were  anxious  while  taking 
the  test  because  it  was  timed  and  because  they  thought 
it  would  affect  their  grade. 


TRiJ^onses  to  the  statements  in  the  remainder  bf  the 
questionnaire  are  reported  with  the  four  response 
categories  collapsed  Into  two:  agree  and  disagree. 
Percentages  of  responses  In  each  of  the  four  original 
categories  may  be  found  In   Appendix  C. 

•The  teacher  was  sufficiently  concerned  about  this  long 
delay  that  she  mentioned  it  to  th©  investigator  before 
the  third  testing  session,  asking,  "Shall  I  tell  then 
to  begin  writing  Immediately?" 

••This  leaves  only  about  20»  of  the  students  who  re- 
ported both  that  they  finished  In  time  for  re-reading 
and  that  they  actually  re-read  their  papers. 
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Commentary  It  appears  that  this  group  of  writers  is 
less  concerned   about  leaving  time  for  correcting  and 
■  ore  concerned  about  taking  time  for  planning.  In 
using  the  short  time  allowance  (30-minutes)   to  best 
advantage,   70*  of  the  students  spent  the  first  5-10 
■Inutes  thinking  about  what  to  write.  Consequently, 
some  of  these   planners  felt  somewhat  rushed  when  they 
actually  started  writing.     Half  of  the  students  did  not 
have  the  time  to  re-read,  and   30»  of  the  students  did 
have  the  time  but  did  not  re-read. 

Responses  to  the  Test  Subject.     Only   11.  1»  of 
the  students  indicated  tHey  wrote  better  than  usual  on 
the  timed  writing  test.     A  large  number   (60. 6J) 
reported  they  had  difficulty  choosing  which  one  of 
several  possible  experiences  to  write  about,  indicating 
the  prompt  opened  access  to  a  wealth  o  f '^po  ssi  b  i  1  i  t  i  e  s  . 
However,   (31. 3»)  noted  that  they  had  diffLculty  think- 
• ing  of  any  experience  to  write  about,  indicating  the 
prompt  did  not  stimulate  recall  of  "first  time" 
■emdries.     A  slightly  larger  proportion   (36. 7»)  indi- 
cated  they  enjoyed  the  assignment  more  than  they  usu- 
ally enjoyed  writing.     However,  the  meaning  of  this 
response  is  ambiguous:   It  is  not  clear  whether  these 
students  liked   the  prompt  better  than  their  usual  writ- 
ing assignments,  whether  they  liked'  the  limited  amount 
of  time  that  they  had  to  spend  writing,  or  whether  they 
responded   positively  to  some  other  aspect  of  the  test 
situation.     However,   62.2%  of  the  students  Indicated 
they  would  like  to  write  about  the  prompt  when  not 
being   tested.     This  percentage  Is  close  to  the  number 
of  students  who   felt  they  would  have  treated  the  prompt 
differently  If  they  could  have  done  the  writing  at 
home. 

Host  of  the  stydents  (78.811)   Indicated  that  they 
had   never   written  about  a   first  time  experience  before. 
More  than  half  (5»».5»),  however.  Indicated  that  they 
had   talked  with  others  about  the  particular  experience 
they  had  chosen  to  write  about. 

Commentary.     This  latter   finding  supports  others 
described  elsewhere.     It  suggests  that  this  invitation 
to  recall  a  first  time  experience  encourages  students 
to  draw  on  anecdotes  that  are  readily  available  in 
memory  and  may  already  have  been  rehearsed  orally.  It 
would  be  useful   to  investigate  how  the  properties  of 
different  prompts  either   function  to  constral^n  to 
enable  d I fferent  assessment  populations  In  selecting 
possible  experiences  to  write  about. 

Writers'   Self-Profiles:  Other  Wrltliig  Exper  l- 
ences.     A  majority  of  the  students  considered  them- 
sel ves  good  writers  (65. 7S).     Only  16. 2>  Indicated  that 
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tney  preferred  writing  academic  essays  or  non-fiction 
reports  rather  than  short  stories  or  personal  essays 
about  their  own  experiences.     While  most  of  the  stu- 
dents (63. 7X)  did  not  keep  a  diary  or  Journal  of 
experiences  or   thoughts,  nearly  half  (M8.5X)  indicated 
that  they  souetimes  wrote  stories,  poems,  or  other 
pieces  outside  of  school  when  the  writing  was  not 
assigned.     About  a   third  of  the  students  (28.6%)  indi- 
cated  they  did  not  have  much  prior  experience  with 
timed  writing  tests.     A  majority  of  students  (67. 7X) 
did  not  think  the  test  was  a  good  measure  of  their 
abilities  as  writers.     Only  39.^*  thought  the  test  dif- 
fered  from  other  timed  writing  tests  they  had  taken. 
In  addition,   68. 6X  of  the  students  thought  the  test 
required  the  kind  of  writing  they  had  to  do  for  other 
high  school  English  assignments. 

Commentary \     These  observations  suggest  two  possible 
TrTteFpr etations  of  the  way  students  perceived  the  task 
in  relation  to  their  school  writing  assignments. 
First,   the  students  may  have  felt  that  both  this  timed 
writing  test  and  their  typical  school  assignments  are 
poor  measures  for  assessing  their  writing  ability. 
Specifically,   students  may  have  thought  that  this  test 
prompt  did  not  provide  a  good  measure  of  their  writing 
ability  and,  because  the  test  prompt  was  similar  to 
other  assignments  they  receive  in  high  school,  school 
writing  assignments  are  also  poor  measures  of  their 
writing  ability.     Alternatively,   the  students  may  have 
been  responding  to  the  timed  aspect  of  the  test  situa- 
tion when  they  reported  that  it  was  a  poor  measure  of 
their  writing  ability.     And   they  may  have  be^n  respond- 
ing to   the  subject  when  they  reported  that  the  test 
required  the  kind  of  writing  required   for  other  high 
school   English  assignments.     If  they  were  responding  to 
the  timed  aspect  of  the  test,  the  students  did  not  mean 
to  give  a  pejorative  evaluation  of  their  high  school 
assignments  when  they  reported  that  this  test  was  not  a 
good  measure  of  ability. 

The  latter  interpretation  seems  more  acceptable 
given  the  fact  that  a  majority  of  the  students  (72.7*) 
believed  they  would  have  written  very  differently,  and 
would  have  done  a  better  Job,   if  they  had  been  allowed 
more  time  to  write.     The  fact  that  a  majority  of  the 
students  (62,2t)  indicated  they  would  like  to  write 
about  the  prompt  when  not  being  tested  lends  additional 
support  to  the  latter  Interpretation.     Finally,  the 
teacher  reported  that  the  prompt  for   this  timed  writing 
test,   calling  for  expressive  writing  from  personal 
experience,   was  similar   to  several  other  classroom 
assignments  given   to  this  particular  population  of  stu- 
dents*    The  available  data  suggests,   then,   that  the 
students  -rere  responding  to   the  nature  of  the  writing 
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occasion   (a  tiraed   test)   rather  than  to  the  "kind"  of 
writing  required  by  the  test  when  they  called  the  test 
a  p\Dor  oeasure  of  their  ability. 
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RELATIONSHIPS  BETWEEN  SELECTED  VARIABLES 
AND  SPECIFIC   QUESTIONNAIRE  ITEMS 


When  rtsponses  to  specific  questionnaire  items 
**ere  examined  in  relation  to  variables  such  as  verbal 
skill,   attitude,  sex,   proficiency,  and  holistic  score, 
additional  interesting  information  was  found. 

Verbal  Skill  Rating  and  Questionnaire  Items 

With  one  exception,  no  significant  relat ionsnips 
were  found  between  students'   verbal  skill  rating  and 
individual  questionnaire  items.     The  only  significant 
relationship  found  was  between  verbal  skill  rating  and 
Item   18  (Tau     s   ,23,   P  <   .05).     As  the  teacher's  rating 
of  the  student's  verbal  skill  increased,  students 
tended  to  agree  with  the  statement,   "I  consider  myself 
a  good  writer."   Thus,   student  self-perception  tended  to 
agree  with  teacher  perception.' 

Attitude  Rating  and  Questionnaire  Items 

Significant  positive  relationships  were  found 
between  the  teacher's  rating  of  student* attitude  toward 
academic  achievement  and  several  questionnaire  items. 
As   attiwude  rating  increased  there  was  greater  agree- 
ment with  the  following  statements: 


3.       I  did  not  have  enough  time  to  write. 

(Tau     s   .26,   p  <  .D5) 
c 

8.       I  did  not  have  time  to  re-read  or  proofread 
m  y  p  a  p  e  r  . 

(Tau     X   .20,   p  <  .05) 
c 

10.  I  believe  I  would ^have  written  very  dif- 
ferently about  this  topic  if  I  could  have 
taken  the  assignment  home  to  complete,  or  if 
I  had  several  days  to  work  on  it  in  jj'lass. 

(Tau^  r   .17.  P  <  .051 
c 

11.  I  believe  I  could  have  done  a  better  job  with 
this  assignment  if  I  had  been  given  an  hour 
to  v(.r:ite. 

(Tau^  X   .27t  P  <  .05) 
c 


Significant  negative  relationships  also  were  found 
between  attitude  and   several  questionnaire  items.  As 
•attitude  rating  decreased,   there  was  greater  agreemant 
with  the  following  items: 
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9.       AlthoLi?rt   I   finisned  b^^'ore  the  ti.T.e  ^as  up,  I 
did  not  ra-read  my  ^.apsr . 


(Tau     =  -.13,   p  <  .05) 
c 
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I  think  the  timed  writing  tsst  we  took  is  a 
good  maasure  of  uy  abilities  as  a  writer. 
(Tau     =   -.23.   P  <  '05) 


In  general,   then,   those  students  who  are  perceived 
by  the  teacher  as  being  conscientious  about  their 
acadstjic  achievernan  t  and  positive  about   their  work  ara 
the  ones  who  felt  tha  30-rninuts  test  did  not  allow  tham 
enough  tinia  to  d  e.uon  s  t  r  a  t  a  ,  the  i  r  usual  or  best   level  of 
performance.     These  are  students  who  would  re-read, 
given   tima,   and  who  are  lively  to  have  set  more  complex 
goals  for  themselves  had  they  been  working  at  homa  with 
unlimited  time.  .  Students  with  poorer  attitudes  toward 
acnieving   in  school  did  less  re-reading,   even   if  there 
was  tima,   and  ware  generally  satisfied  that  the  timed 
test   adequately  measured   their  abilities. 

3ex  and  Questionnaire  Items* 

Males  and  females  appeared  to  differ   in  response 
to  the  time  demands  of  the  task.     Altnough  most  stu- 
dents indicated  they  had  enough  time  to   finish  the 
task,   females  ten-d^  to   indicate  the  opposite  more  fre- 
quently than  males;   43. 9t  of .  the  females  report,ed  that 
they  did  not  have  enough  time  to  write,   compared  with 
only  21. 4X  of  the  males.     This  association  between  sex 
and  response  to  item  3  wa s  ,  s i gn i f ic an t   ( $ch i  sup  2$  = 
4.45',   P  <  .05). 

Significant  associations  were  also  found  between 
sex  and  responses  to  questionnaire  item  20  ("I  keep  a 
diary  or  Journal  of  my  experiences  or  thoughts.")  and 
item  21  CI  sometimes  write  stories  or  poems  or  other 
pieces  outside  of  school,  even  though  the  writing  was 
not  assigned.").  A  journal  or  a  diary  was  kept  by 
43. 9X  of  females,  but  only  14.3*  of  males  reported  that 
they  kept  a  diary  or   Journal   (c^i^  =  8.51.  p  <  ^  ^ 

.05).     Similarly,    57.9%  of  females  but  only  35. 7X  oi 
males  reported  that  they  wrote  unasslgned  pieces  out- 
side of  school   (chij  =  3.92  p  <  .05). 


» rnFs  analysis  was  Bonducted 
categories  of  responses  on 
categories   (disagree  and  agr 


after  collapsip-g  the  fojr 
the  qutstionn/aire  into  two 
ee )  . 


Proficiency  and  ^uastionnaire  Items* 

<4nen   the  qj  es  t  i  onn  a  i  r  e  data  ware  exanined  in  terms 
of  tne  proficiency   level   of  tne  students,   several  very 
stri;<in8  and  consistent  patterns  appeared.  First, 
there  were  significant  associations  between  proficiency 
level   and  responses   to  q  j  e  s  t  ionn  a  i  r  e  items  3  .    ,  6  ,  7  ,  8  , 
and  9.     Tnese  associations  seem  to  suggest  that  profi- 
cient writers   1)   used  a  substantial  aaount  of  planning 
ti.-ne,   2)   felt  rushed  when  tney  actually  started  writ- 
ing"  3)   felt   tney  didn't  have  enough  time  to  write,  and 
4)  did  not  have  time  to  reread  their  papers. 

Tnere  was  a  significant  relationship  (chi  =1.51, 
p  <  .05  between  proficiency  and  respoTise  to 

the  statement:  "I  spent  a   lot  of  time   (at  least  five  or 
ten  minutes)   thinKing  about  what  I  wanted  to  write 
before   I  started."   A  majority  of  the  proficient  writers 
(77.01.)   agreed  with  this   statement  compared  to  52. OX  of 
tne'less  proficient  writers.     At  least  part  of  the 
additional   planning  time   that  proficient  writers  seem 
to  spend  was  taken   up  in  deciding  which  of  several  pos- 
sible experiences   they  s  no  u  Id- .wr  1 1  e  about.     Fo"  exam- 
ple,  67. 6X  of  tne  proficient  writers,   compared  to  40. Ot 
of  the  less   proficient  writers,   agreed  with  the  state- 
ment: "I  had   a  hard  time  choosing  which  one  of  several 
possible  experiences   I  should  write  about."   (  ahi2 

=  4.85,   p  <   .05).     Writing  proficiency  seems  to  be 
associated  with  taking  5-10  minutes  of  planning  time  to 
consider  a   range  of  alternatives  accessible  in  memory 
in  selecting  a  specific  experience  to  write  about. 

jt 

Because  proficient  writers  take  so  much  time  to 
plan  their   composition,   they  often  feel  rushed  when 
they  finally  co.mmence  writing.     Proficiency  was  signi-  . 
ficantly  associated  with  agreemen.t  with^item  7, 
"Because  I  spent  so  much  time  planning,   I  felt  rushed 
when   I  actually  started  writing."   (chi2  =  8.79, 
p  <    .05).      This  association  is  suggested  by  the  fact 
tnat  43. 2X  of  p ro f i c i en t . wr i t e r s  agreed  with  this 
statement  compared   to  only   8*  of  the  less  proficient 
wri  ter  s  . 

The  extended   planning  time  typical  among  these 
proficient  writers  may  also  contribute  to  their  feeling 
about   the  amount  pf  time  necessary  to  adequately  com- 
plete the  assignment.     Proficiency  was  significantly 
associated  with  agreement  with  item  3:   "I  did  not  have 
enough  time  to  write."   (  chl^  =  8.79.  P  <  •05). 

"•TiTir'ana lysis  was  conducted  after  collapsing  the  four 
categories  of  responses  on  tne  questionnaire  into  two 
categories    (disagree  and  agree). 
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Proficient  writers  acre  frequently  agreed  wltrh  itea  3 
eoapared  to  less  proficient  writers  (Proficient  s 
43. 2X,  Less  Proficient  s  8.01). 

Finally,  the  extended  planning  time  typical  of 
proficient  writer*  nay  have  prevented  amy  of  them  from 
re-reading  or  proof-reading  their  paperi.     Thepe  was  a 
significant  association  between  proficiency  and  agree- 
ment with  item  8:   "I  did  not  have  time  to  re-read  or 
proof-read  my  paper."  (   chi,  *  8.03.  P  <  .05). 

Some  59. 5X  of  proficient  writers  agreed  with  this 
■tatement  compared  to  2U.0X  of  less  proficient  writers. 

There  is  also  tome  indication  that  less  proficient 
writers  may  be  less  likely  to  re-read  their  papers. 
For  eiaaple,  52.01  of  the  less  proficient  writers 
agreed  with  item  9,  "Although  I  finished  before  the 
time  was  up,  I  did  not  re-read  my  paper.",  compared  to 
only  23.01  of  the  proficient  writers  (  chi2  * 
6  1U,  p  <   .05).     This  association  may,  however,  be  due 
to  the  greater  number  of  less  proficient  writers  who 
had  sufficient  time  to  re-read  their  papers  (less  pro- 
ficient «  76. OX,  proficient  «  40. 5X). 

In  addition  to  the  associations  that  exist  between 
proficiency,  planning  time,  and  the  consequences  of 
taking  5-10  minutes  to  plan  one's  composition  during  a 
30-Binute  test  period,  there  was  an  association  between 
proficiency  and  student's  attitudes  about  how  addi- 
tional time  to. write  would  have  influenced  their  writ- 
ing oerformance.     Proficiency  was  positively  associated 
(Shi!  «  5.91,  p  <  .05)  with  the  belief  that  the 

final  product  would  have  been  very  different  if  stu- 
dents had  leveral  days  to"^  complete  the  task.     Of  the 
proficient  writers  79.7*  agreed  with  this  itateaent 
(item  10)  compared  to  only  18. IX  of  the  less  proficient 
writers.     Proficiency  was  similarly  associated  (  chi^ 

*  18.75,  P  <  .05)  with  the  belief  that  the  filial 
product  would  have  been  better  if  students  had  had  an 
hour  to  write.     Of  the  proficient  writers  83. 8X  -Sreed 
with  this  statement  (Item  11)  compared  to  36. OX  of  the 
less  proficient  writers.     Proficient  writers  believed 
that  additional  time  would  have  changed  and  benefited 
their  writing  performance. 

Comraentaryt     The  pattern  of  results  from  investigating 
the "relation ship  between  proficiency  and  questionnaire 
items  suggests  that  a  30-minute  timed  writing  test  may 
not  adequately  measure  the  ability  of  proficient  writ- 
ers      Placing  a  very  short  time  limit  on  the  writing 
task  may  create  a  ceiling  effect  on  students'  perfor- 
mance.    Even  though  their  writing  was  generally  supe- 
rior, proficient  writers  report  that  they  could  have 
done  even  better  if  they  had  had  more  time.     Tne  fact 
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competent  writers.     Further  empirical  support  for  this 
first  flndins  in  additional  studies  would  '"""^J^^'^ 
Short  timed  writing  tests  are  likely  to  severely  trun- 
cate  the  range  of  performance  elicited  ^ 
assessment,   although  good  writers  -^ll/^.^J.^*"'!'  , 
top  scores  on  a  short  test.     The  second  ^  "^J^f/^i;*' 
soae  question  about  whether  the  students  in  this  study 
received  high  holistic  scores  because  they  f^"" 
erally  better  writers  or  because  they  had  previously 
written  about  the  topic.     It  «ay.   at  a  later  date,  be 
profitable  to  explore  the  effects  of        °%  "       J*"  , 
with  a  topic  on  writing   performance.     The  third  finding 
indicates  simply  that  good  writers  are  aware  of  ^heir 
ibilities.     It  does,  however,  raise 

the  effects  of  s  e  1  f -e  v  al  ua  t  ion  cr.  writing  ^^^1^^^'  t:*" 
does  a  positive  concept  of  oneself  as  a  writer  facili- 
tate writing  performance? 

Prompt  Version  and  Mode  and  Planning 
Time  Questionnaire  Items 

Table  9  summarizes   the  results  of  the  question- 
naire data  by  prompt  version.     Many  items  were  not 
as  ociated  with  differences  between  versions  Differ- 
enees  did  appear,  however,   in  several  response  items, 
:rd  this  suggest;  an  interesting  effect         Promp  s  on 
attitudes  toward  the  test  experience  as  well  as  an 
?5ttJesttng  effect  of  prompts  on  the  writing  process 
Itself. 

For   example,   -ore   students  who  -'"^  /[.^"P^,, 
version   B  seemed  to  think  the  test  was  a  good 
of  fch.lr  abilities  as  a  writer  (Item  24).     Of  the  stu- 
dents iho  wo  e  on  Prompt  version  B.   UO.8%  agreed  with 
Jhis  statement  compared  to  only  2U.5I  of  students  who 
wrote  on  Prompt  Version  A. 

AH  of  the  other   important  differences  between 
versions  were  related   to  the  way  students  described 
their  writing  process   (Items  '».6.-"^J>-     *  J  ^at^/ 
Quency  of  students  who  wrote  on  Version  A  indicated 
that  they  spent  a  significant  amount  of  time  planning 

h     r  ?espoSse.     More  -students  who  wrote  -  Ve'-sion  A. 
for  example,   reported  that  they  spent         l.«"J  5-  0 
ainutes  thinking  about   what  they  wanted  to  -"-Jte  (Ver- 
3  on  I  X  79.61.   Version  B  .  62.01).     More  students  who 
■  wrote  on  Version  A  also  reported   that  they  had  diff  - 
^ulty  Choosing  which  of  several  P° "  J '>1%"P;^,^«""' 
wr^te  about   (Version   A  »  75.51.   Version  B  x  H6.0%). 
?h;ap;a?ent  discrepancy  in  a.ount  of  P^^"" J"*  ^i:*,, . 
beW-en  prompt  versions  is  also  supported  by  students 
.J:^;;nse'to  ?tem  7.     More  students  who  «^°^«  !" 
A  felt   rushed  when  they  actually  started  writing  (Ver- 
sion  A   =  ilO.at,    Version  B  =  28. OJ) 

n  rr :  ■ 
.'-O.J 
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In  general,  students  who  wrote  for  a  less  speci- 
fied audience  (Version  A)  more  frequently  spent  a  sub- 
stantial aiiount  of  time  planning  what  they  were  going 
to  write.     Students  who  wrote  for   a  acre  specified 
audience  (Version   3)   less   frequently  spent  a  substan- 
tial amount  of  time  planning.     This  difference  is 
illustrated   in  Figure  1. 

It  is   important  to  note,  however,   that  despite 
these  process  differences  there  was  no  significant 
difference  between  versions   in  the  final  quality  of  the 
Product  as  measured  by  tolal   holistic  score  (Hann- 
Snltney  U  =   U80.5,   z  «  -.85,   P  >   .05).     Of  the  99  stu- 
dents that  completed  both  the  w.'iting  test  and  the 
questionnaire,   the  aean  holistic  score   for  Version  A 
students  was  6.09   {Ns*»9):   for  Version  B  students  the 
mean  hol<5tic  score  was  6.28   {N  =  50).     On  the  ^^[^^^ ' 
at  least,   these  versions  appear  to  be  similar  with 
respect  to  the  holistic  scores  that  they  produce. 

In  addition  to  an  analysis  of  the  main  effects,  an 
analysis  of  the   interaction  between  prompt  version  and 
planning  time  on  writing  performance  was  conducted. 
Table   10  indicates  that  students  who  wrote  on  Version  A 
and  students  who  wrote  on  Version  B  also  performed 
somewnat  similarly. 

In  addition  to  this  two-way  analysis  between 
prompt  version  and  planning  time,   a  three-way  analysis 
was   conducted  between  prompt  version     planning  time, 
and  re-reading.      Table   11   shows   the  8  possible 
tlons  of  these  three  variables,   the  means  and  variances 
of  the  holistic  scores  for   each  combination,   and  the 
number  of  students  that  fell  Into  each  group. 

In   the  writing  performance  of  Version  A  students, 
re-reading  seemed   to   Interact  with  pi  a nn 1 ng  t Ime .  Ver- 
sion A  students   in  Group  *»  who  re-read   their  papers  and 
spent  5-10  minutes  planning  had  a  higher  mean  holistic 
score   (X   s  6. '12)   than  students   In  Group   2  who  re-read 
their  papers  but  did  not  plan  (X  x  5.67).  Apparently, 
re-reading  only  had  an  effect  for   Version  A  students 
if,   in  addition,   they  spent  5-10  minutes  planning. 

Not  taking  time  for  re-reading,   among  Version  A 
students,  also   Interacted  with  planning  but   In  the 
opposite  direction.     That  Is,   Version   A  students  In 
Group   1  who  did  not  re-read   their  papers  and  who  djd 
not  sp-nd  5-10  minutes  planning  had  higher  mean  holis- 
tic scores   (X  «  6.43)   than   students  In   Group   3  who  did 
not  re-read  and  did  plan   (X  =  5.93).     Not  re-reading, 
for  Version  A  students,  was  associated  with  higher- 
holistic  scores  only  for   those  students  who  also  did 
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A-nong  Version  B  students,   re-reading  also 
interacted  with  planning  time.     Students  In  Group  5  who 
re-read  and  did  not  plan  had  higher  aean  holistic 
scores   (X  =  6.29)   than  students  in  Group  8  who  both 
re-read  and  planned   (X  =  6.00).     Apparently,  re-reading 
only  had  a  positive  effect  for  Version  B  students  if. 
In  addition,  they  did  not  plan. 

Not  re-reading,  aaong  Version  B  students,  also 
Interacted  with  planning  tlrae.     Version  B  students  in 
Group  7  who  planned  but  did  not  re-read  had  higher  mean 
holistic  scores   (X  =  6.55)   than  students  in  Group  5  who 
neither  planned  not  re-read  {X  =  6.25). 

Additional  findings  of  Interest  between  prompt 
version  and  questionnaire  items  Involve  the  relation- 
ships  that  were  found  with  the  mode  of  writing  that 
students  selected.     More  students  who  wrote  narratives 
(73.7%)   tnd  journals  (75. OX)   indicated  that  they  spent 
5-10  minutes  planning  than  students  who  wrote  letters 
(55.6X)o     More  students  writing  in  non-narrative  forms 
finished  and  re-read  their  papers   (letter  s  83. 3X. 
journal  and  essay  =   100X)   than  did  those  students  writ- 
ing a  narrative   (64. 5X).     Overall,   those  students  who 
wrote  letters  generally  felt  they  could  have  done  a 
better  Job  with  more  time   (83. 9X)   compared  to  those  who 
wrote  narratives    (68. MX)   or  journal  entries   (50. OX). 

Regarding  opinions  about  the  writing  task,  all  of: 
the  students  choosing  non-narrative   forms   (N»23)  did 
not  think  they  wrote  better  on  this  assignment  than 
they  usually  do.     This  compares  with  only   1«.4X  of  the 
students  who  cho^e   a  narrative  mode. 

These  response  frequencies  suggest  that^  selecting 
a  non-narrative  mode,   particularly  the  letter  form,  had 
the  effect  of  making  these  students  wonder  whether  the 
task  enabled  them  to  show  their  true  ability. 

As  reported   elsewhere,   although  students  generally 
thought  that  the   task  was  not  a  good  measure  of  their 
writing  ability,   letter  writers  reported  this  more 
often  than  did  students  who  wrote  narratives  or  jour- 
nals  (letter  =  77.81.  narrative  =  67. U.   journal  = 

50e0X)  o 
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One  of  the  acre  Interesting  results  from  this 
study  Is  the  observation  that  differences  In  prompt 
version  did  not  produce  a  difference  In  performance  as 
measured  by  total  holistic  score  means.     The  mean  per- 
formance of  students  who  wrote  on  Version  A  was  nearly 
identical  to  the  mean  performance  of  students  who  wrote 
on  Version  B.     This  would  seem  to  suggest  that  increas- 
ing the  degree  to  which  the  writer's  audience  and  rhe- 
torical purpose  are  specified  has  no  effect  on  the 
quality  of  a  timed  writing  performance.     There  are 
several   possible  explanations  for  this  result,  however, 
that  must  be  considered.     Each  raises  an  additional 
research  issue  that  should  be  explored  before  conclud- 
ing  that  audience  specification  has  no  effect  on  the 
quality  of  a  student's  performance. 

It  should  be  noted,   first,   that  the  two  versions 
are  not  extreme  examples;   that   is,   they  are  not  at 
opposite  ends  of  an  audience  specification  continuun. 
In  Version  A,   the  audience,   though  general,  is 
nevertheless  mentioned   (i.e.,   "the  reader").  Moreover, 
including  a  clearly  spaclfled  communicative  purpose 
(i.e.,   "Write  atsout  this   'first  time'   so   that  your 
reader  understands  how  you  felt  and  why  this  experlcrce 
stayed  in  your  memory")   is   likely,   by  Itself,   tc  con- 
strain the  set  of  possible  audiences  and   the  sense  of 
rhetorical  purpose.     Implicit  in  this  task  is  the  idea 
that  one  should  write  to  an  audience  that  can  under- 
stand and  appreciate  a  writer's  particular  "first  time" 
experience.     In  Version   B,   the  audienoe  is  only  vaguely 
specified   (i.e.,   "Imagine  that  you  are  writing  to  some- 
one who  has  Just  had  a  similar  experience,  or   to  some- 
one who  is  about  to  have  such  an  experience")  The 
writer  does  know  something  about  the  reader;  he  or  she 
has  or  will  have  a  similar  experience,   but  no  more  than 
that.     A  much  more  detailed  specification  of  the  audi- 
ence is  clearly  possible.     The   writer  could  have  been 
asked,   for  example,   to   explain   the  experience  to  a 
retired  man,   in  his  sixties,   who  was  recently  widowed 
and  who  will  shortly  share  a  similar  experience. 
Clearly  what  is  needed   is  additional  research  on  pairs 
of  prompts  in  which  the  difference  in  audience  specifi- 
cation  is  made  more  evident.     The  failure  to  find 
differences  in  the  quality  of  writing  between  these  two 
versions  may  have  been  due   to  the  fact  that  they  both 
rep  esent  Middle  range  examples  on   the  continuum  of 
audience  s psc 1 f 1 c a t ion . 

Though  there  was  no  association  evident  between 
the  assigned  prompt  and  holistic  score,   the  effects  of 
the  prompts  are  clearly  associated  with  the  students' 
choice  of  mode.     Equal  numbers  of  students  wrote  on 
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.aeh  vrslon.  but  Version   B  students  produced  aore 
each  w-rsion,     u  ....       These  students  produced  20 

alternate  modes  of  wrltins.     incac  s  ,   .n*-rio<  l 

Alternate  -node  papars  (16   letters.   3  Journal  entries.  1 
tssay)     While  students  -ho  -rote  on  Version  A  produced 

7  alternate  aode  papers   (5   letters.   2  journal 
TniMs)       Mo^t  0     the'-ore  personal   fdrms  occurred  in 
response  to  Version  B. 

Although  mean  holistic  scores  did  not  »U"JJ^- 
cantly  differ,   there  -ere   clear   differences   in  the 
olannLrproc;ss.     Decreasing  the  audience  specifica- 
tion seemed   to  produce  .n   increase  in  planning  time 

test  requiring  impromptu  -riting. 

There  is  a  second  possible  explanation 

population  Jlffe-ed   1"   oer  representative  sample 

for  «rb,r5klll.  attitude,  .nd  holUtlc  ,  =  or.  are  all 

!,°e-ed   to«rd,   the  nigh  end  f  fJ^^i^J"?  ' 

11   5J  of  the  papers,  received  one  of  the  tv^o  nign.s 
illlUii  ,oor»  C7.8).     Only  5.«  =f  the  P^P"'.^^^,^ 
received  one  of  the  two  lowest  scores   C2-3).  because 
scor       on  the  performance  J' ff '  ' 

topping  out  effect  may  have  .Inlmlied  the  ""8=  " 
5???erences  and  therefore  U.lted  the  I'^'l'''^^^^^' 

^iltltL's'cJrerr^ht'Jir-rll^oh^!":  5J  d^  t-popu- 

Kri°?e^.diiri:irg-.!:d?nh^i'n^?:rL;h:d  or 

'^u:rtti;"trt5lhutlo'n  .ay  he  U.ely  to  pro- 

duce significant  differences  In  prompt  effect. 

A  third  explanation  for  equivalent  mean  scores  for 
the  t«  "o.pt  Versions  Is  compatible  with  the  findings 
interactions  between  '-'""/""""J  "   iJ'f,  I 
'-'"l;  IZlir.l  :tir«'ir  hrroduc-t'per'fSrmaJcrof 
^  :  „'ers"rd'nor-pUniers  In  the  t-ojerslon  groups  may 

-:rthru:h'in'^f:;t:"Se'r.  /  H  "i  1  i  r"- 

i;:i^Lr;rerrlsrar%£^rt"sSo3?ra%oLirirird  'ter. 
lir.  Whether  the  processes  chosen  -ere  natural  to  the 
:  adents  -no  chose  them,   or  -hether  they  /"J  - 

enred  by  the   prompt  version,   and  -hether  these  were 
Tptlma?   process  decisions  for   successful  performance. 

2nl  ■■'O't 


Observing  the  saae  students  over  several  waiting  tasks 
woaid  reveal  whether  soae  kinds  of  tasks  consistently 
elicit  or  require  more  planning  time  than  others,  at 
least  among  students  who  plan,   and  whether  planning 
consistently  produced  better  products. 

We  may   finally  speculate  6n  the  effect  of  the 
observation  that  Version  A,   because  of  fewer  con- 
straints,  seems  less  likely  to  quickly  provide  students 
with  a  good  subject  to  write  about.     Students  who  are 
serious  about  their  writing  and  who  therefore  seek  sub- 
jects they  care  about,  may  be  required  to   spend  more 
time  searching  when  faced  with  Version  A.     (Less  seri- 
ous, more  perfunctory  writers  faced  with  Version  A  aay 
begin  writing  quickly,   having  fewer  resources  for  scan- 
ning and  selecting  appropriate  subjects, —  hence  the 
association  of  lorfer  scores  with  Version  A  non- 
planners.)   Version  B,   on   the  other  hand,   while  allowing 
such  a  search,   see^aad  to   trigger  -more  instant  responses 
in  a  wider  range  of  students.     Students  appeared  to 
quickly  choose  a  subject  which  usually  engendered  suc- 
cessful writing  within  the  given  time  limit.  The 
effect  of  this  difference  may  be  paradoxical:   the  extra 
search  for  meaningful,   appropriate  subject  matter 
engaged   in   by  Version  A  students  may  have"  contributed 
soaetning  to  the  composing  process  that  took  these 
papers  to  tne  highest  level  of  quality  —  a  level  less 
available   to  Version   B  students  whose   subject  cams  more 
quickly  to  mind. 

Tne  implieations  of  such  a  finding,   if        were  to 
be  sonfirmed  by  more  precise  observations  of  the  com- 
posing process  and  analysis  of  the  written  products, 
would  be  minimal   for  general   psychometric  purposes,  but 
would  be   important  for   the  evaluation  of  instructional 
programs  and  for  research  measures  which  seek  to  assess 
differences   in  the  performance  of  able  as  well  as  less 
able  student  writers.     The  large  difference  in   the  fre- 
quency with  wnich  different  rhetorical  modes  were  pro- 
duced by  the  fvo  prompt  versions   is  also  of  interest. 
The  version  with  the  greatest  audience  constraint  (B) 
produced  far  more  papers  of  a  personalized  type  (jour- 
nal entry  or  letter).     It  would  be  useful   to  know  if 
this  Is  due  solely  to  increased  audience  specification 
or   if  it  is  also  a  function  of  the  particular  comrauni- 
eative  purpose   implied  in  Version  B.     Explaining  a 
first  time  experience  to  someone  who  is  ibout  to  or  has 
just  had  a  similar  experience  the  writer  shares  an 
important  adventure  with  his  or  her  reader.     It  may, 
thus,   call   for  a  shift  in  the  register  or  the  form 
selected  by  the  writer.     It  would  be  interesting  to 
know  if  more  personalized   forms  are  also   produced  when 
the  audiense  specified  excludes  peers:   "Explain  this 
first-time  experience  to  a  prospective  employee  who  has 
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just  had  or   is  about  to  have  a  similar  experience." 

Another   finding  of  this  study  seems  also  to 
require  additional   investigation.     The   fact  that  a  sub- 
stantial proportion  of  these  students   (72. 7»)  reported 
that   tr.ey  would  have  written  very  differently  about  the 
prompt  if  they  had  had  more  time  to  work  on  it  raises 
questions  about  the  ecological  validity  of  holistic 
scores  on  a   single  sample  as  an  assessment  measure.  It 
suggests  that  students'   performance  on  a  short,  timed 
writing  test  may  differ  substantially  from  the  condi- 
tions more  typically  associated  with  regular  classroom 
writing  assignments,   at  least   in  this  teacher's  class. 
It  would  be  useful   to  understand  how  performance 
differs  as  the  amount  of  available  writing  time 
increases:   are  different  forms  of  writing  produced  when 
students  have  more  time?     Does   the  quality  of  writing 
differ  substantially?     What   type  of  student  seems  to 
revise  the  most  when  greater   time  is  available?     How  is 
the  writing  process  different  when  greater  time  is 
available?     Which   individual   differences   (grade,  sex, 
vo-rbal  ability,   attitude,   etc.)   seem  to   ^"teract  with 
differences   in  the  amount  of  available  writing  time  and 
seem  to  produce  changes  in  the  quality  of  writing? 

In  general,   students  who  are  likely  to  receive  top 
s-^ores  on   longer  tests  also  receive  top  scores  on  short 
tests.     snort,   timed   tests,   then,  may  be   adequ^ite   in  a 
gross   way  to   identify  proficient  students  or  predict 
success  at   longer   tasks.     However,   they  may  be  inade- 
quate to  distinguish  among  various  levels  of  profi- 
ciency and  hence  may  be  poor  measures   for  evaluating 
writing  programs  which  aim  to  move  writers  along  a 
scale  toward  greater  and  greater  proficiency. 

If  the  purpose  of  an  assessment  is  merely  to  meas- 
ure minimum  proficiency,   short  tests  may  provide  a  suf- 
ficient range  of  performance  among  most  populations. 
But  since  short  tests  may  reveal  only  the  grossest 
differences   in  ability  among  homogeneous  populations, 
improvements  in  writing  ability  over   a  semester  or  year 
course  as  a  result  of  certain   instructional  practices 
may  be  totally  obscured,  with  students  appearing  to 
make  little  or  no  improvement.     Hence,   short  tests  may 
be  poor  research  or   evaluation  measures  for  improving 
instr  uction . 

It  may  be  argued  that  writing  programs  are  not 
making  important  enough  differences  unless  the  differ- 
ences can  be  recorded  by  fairly  rough  measures 
distinguish  between  proficient   and  non-prof ic lent  wri t- 
ing   —  and   hence,   that  short  timed  tests  are  sufficient 
measures  of  program  effect.     But  this   is  tantamount  to 
saying  that   the  only  growth  in  writing  that  matters 
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the  .-novement  froa  an   inability  to   the  ability  to  pro- 
duce minimally  compe tent  wr i ting  during,  a  30-minute 
writing  task.     Tnis  puts  a  ceiling  not  only  on  our 
measure  but  on  our  goals  for   writing  Instruction. 

Me  must  keep  in  mind cf.  cO'ur  aft .  that  all  of  the 
above  observations  are  based  on  reports  by  student?  of 
their  own  writing  processes.     Although  the  teacher 
observation  that  the   class  was  taking  longer  than  usual 
supports  the  self. reports,   the  above  results  should  oe 
viewed  only  as  suggestive  of  questions  which  might  be 
best  answered  not  by  questionnaire,  but  by,direct 
observation  of  individual   students  during  the  act  of 
composing.     Many  such  observations  will  be  needed  to 
determine  whether  the  kind  of  trend  noted  here   is  a 
significant  one.  and  what   its  implications  are  for  test 
design  and  for  our   understanding  of  the  composing 
processes  of  writers  on  'impromptu  timed  writing  tests. 

Because  statistical  tests  of  a.ign  i  f  i  c  ance  were 
employed   in  analyzing  the  data  collected   for. this 
study,   it  should  not  be  concluded   that  the  researchers 
are  committed  only  to  hypothesis   testing.     Rather,  we 
are  convinced  that  the  current  state  of  understanding 
about  the  performance  of  students  on  timed  writing 
tests  is  sufficiently  pr i m i t i ve,  t ha t  a  wide  range  of 
a-neral  observations  needs   to;be  made  in   tjie  hope  of 
generating  testable  hypothesis,  which  in   turn  will  lead 
to  a  -refinement^  of  our  measures.     Many  conclusions 
about  student  writing  ability  and   the  effect  of  dif- 
ferent instructional   programs  are  currently  based  on 
measures  whose  accuracy  and  power   are  subject  to  seri- 
ous question.     He  believe  the  data  6f  this  preliminary 
study,   while  suggesting  few  clear  answers  to  our  Ini- 
tial questions,  strongly  confirm  that  those  questions 
are  worth  asking  and  are  potentially  answerable. 
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n 
% 

Col  Jan 
Totil 
H 
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Tn«  Studtnt  Population  Ujsd  In  Tnis  Study 
by  Cradt  Ltvtl»  5ti,  Sudjfct  and  Stctlon 


Adyanetd  Coap. 

Stctlon  1 
Jri.  &  Sra. 


12 
(10, 


10 

(3.3) 


22 
(19.35 


English  Lit. 
Stct,   1       Sact.  2 
Jrj«  Jpj, 


12 
(10, 


21 
(18, 


5) 


33 
(28.9) 


13 
(11. 


Gtntral  Cngllsn 
Sact.  \ 
Soph. 


1  1 

(9.6) 


19 
(16.7) 


32 
(28.  1  ) 


16 
( 1  *  .  0 ) 


27 
(23.7) 


Total 


48 
(42.  1) 


65 

(57.9) 


1  14 
(100,0) 


Tablt  2 

Distribution  of  Vtrbal   Sxill  Ratings 


BtlOrf 

3 

(2.6) 


aradt  ttvtl 
or  ftt low 

39 
(3^.2) 


Gradt  Ltvtl 
or  Abo vt 

53 

(46.5) 


i^tll  Abovt 
Gradt  Laval 

17^ 

(14.9) 


Missing 
Casts 

2 

(1.8) 


Total 
114 

(100.0) 
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Tabit  3 


Distrioutlon  of  Attitudt  Riting.^ 


Cs tr tati  y 

9 

(7.9) 


Mildly 

25 
(21.9) 


.Mildly 
Pcsi tl V 

(38.6) 


£Ktrt«f ly 
Po3l tivt 

3^ 

^  (29.8) 


His  a  ing 
Casts 

2 

c  1  -  a ) 


Total 

(100.0) 


N  0 

t  (0.0) 


Tablt  H 

Distrlbuticn  of  Total  Holistic:  Scorts 
3*  5  6  7  8  Total 

2  13  10  117  21  21  1  m 

(1.8)       (1H)       (8.8)       (111.2)       (13. «l)       (18.11)  (100.0) 
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Tablt  5 


itanj  *nd  Standard  DtvlationJ  of  Proflcltnt  Studtnts' 
and  Ltss   Proflc'ltnt  SCudnnti*   Total  HoUitlc  Scoras^ 
by  Class,  Stctlon  and  Gradt  L«v«l 


PIOFICIEMT  STUDENTS 


Advanctd  Coap* 
Stctlon  1 
Jr s .   \  3r J • 


enillsh  Llt« 
Stctlon   1  Stctlon 


Jr  a  • 


Jri, 


Total 


LESS 

PIOFICIEMI  STUDENTS 

Gtntral  English 
Stctlon  1 
>oph « 


6.6i| 
1 .0*19 
22 


6.%9 
1  •  25 
33 


6.111 
1.13 
32 


6.50 
1.21 
87 


5.  19 
1.11 
27 


Tablt  6 

Fr tq Li en-c  1 1 s   for  tnt  Olfftrtnt   4odts  of 
irfrltlni  ProdJCtd  by  Prioapt  A  and  Proapt  fl 
(31vtn  In  Pt^ctntatts) 


Proapt  A 
% 

n 

Prompt  B 
% 

n 


Narratlvt       Lttttr       Journal  Essay 


87.7 

(50) 


(37) 


8.8 

(5) 


28.1 
(16) 


3.5 
(2) 


5.3 
(3) 


,  0.0 
(0) 


1  .  8 

(1) 


Raw 
Total 


50.0 
(57) 


50.0 
(57) 


Total 


76.3 
(37) 


..18.^ 
(21) 


(5) 


0.9 
CI) 


100.  0 
(  1  M) 
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rabl«  7 

StK  by  .iodd  Frtqutnclts 

Narr«tiY«  ^lodt            llon-J<arrativ«  .ioJt 

Lttttr       Journal  Essay  Total 

ftaalts 

"i               66.7                 2^.2            7.6  1.5  57.9 

a                (411)                  (16)            (5)  (1)  (66) 

:4  a  1 1  a  « 

i  39.6  10.6  0.0  0.01  32,} 

.1       '  .(43)  (5)  (0)  (0)  <4r8) 


Tablt  a 

.^ftan  Total  Holistic  Scorts  by 
Fr&flcltncy  and  Hodt  5tltctlon 

Prof Icltnt  3s       Ltss  Prof Icl tnt  3s 

Marrat lyt  Hod t 

J  6.48  5*09 

S. D.  1.11  1-19 

N  65  22  , 

i<on«'Warratlyt 
j^odt 

f  6.55  5.60 

^S.D.  1 .30  0-55 

M  22  5 
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No 
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No 
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7* 
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APPENDIX  A:    WRITING  TEST 


STUDENT  NAME_ 


TOPIC 


Student  Code: 


{«s  you  normilly  write  1t  on  English  papers) 

-r'-T~r-r-T'~r-T-r 


2 


InstrL,-.tions  to  Students; 

1.    Please  write  your  nane  in  the  approprUte  blank. 

Look  briefly  »t  the  test  sheet  attached.  '  You  have  either  Topic  1 
or  Top^c  2       Write  the  nunber  of  your  topic  In  the  space  above  beside 
the  word~^opic." 

To  fin  In  your  student  code,  please  put  the  following  i-'or-afor  In 
the  appropriate  blanks  above: 

blank  1:    The  period  you  have  English  with  f«s .  MCelly  (3.5,5,  or  6 

blanks  2  -  5-  Please  use  the  last  four  digits  of  your  phone  nu-ber 
blanks  Z  ^^ea^^^  ^.^^^^       ^^^^  ^^^^^  ^  whiche.e-  yt. 

prefer,  to  be  your  own  Individual  cede  nurfce-. 

blank  6:    Pleise  write  the  nu^b^rl    If  ycj  are  fe-i>; 

please  write  the  number"  if  you  are  male. 

hiAink  7  and  8-    Please  leave  these  spaces  e-cty.    U  yoi-  cop 
bl«n.     and  H.    ^      ^^^^  ^^^^  additional  sheets  of  pape;;.  Please 

leave  room  for  these  two  digits  to  be  iiiti  by 

your  tuacher. 

One.  vou  'lave  -reated  your  code,  minus  the  last  two  digits,  please  cor. 
?t  into  the  Ippropriate  blanks  on  the  attached  test  sheet.    DC  NC^  V^. 
YOUR  She  on  the  test  sheet.    Your  code  will  assure  that  yo"^ 
leturned  to  you!  but  your  reader  will  not  know  who  you  are  wMle    e  .r 
she  Is  scoring  your  paper. 

the  top  righthand  corner  of  each  page  you  write  on. 
Please  us.  pen  or  dark  pencil  so  your  paper  can  be  duplicatei  if 
necessary. 

If  you  finish  writing  before  the  time  Is  "P-  Pj"",:!;^?^^:;;Vc^ 
look  out  the  window  or  stare  Into  space,  so  that  o.he.s  who  ire 
done  can  continue  writing  without  distraction. 

write  and  little  time  for  tdltlng. 
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APPENDIX  A:    WRITING  PROMPTS 


Students  were  given  one  of  the  following  two  versions  of  the  prompt. 

VERSION  A 


Think  or  an  occasion  when  you  experienced  something  for  the  FIRST  t:*'£.  It 
may  hive  been  something  you  late  came  to  do  more  easily  or  sonet^inc  yoj  now 
take  for  granted;  but,  for  some  reason,  your  first  ti"e  was  r>e-'orab1e.  W'^ite 
about  this  "first  time"  experience  so  that  your  reader  can  understand 
your  feelings  and  why  this  memory  has  Stayed  with  you. 

You  may  write  an  essay,  a  short  story,  a  letter,  or  a  journal  entry  as  t 
way  of  re-telling  your  experience. 


VERSION  3 

Think  of  an  occasion  when  yoy  experienced  something  for  the  FIRST  TIKE. 
It  may  have  been  something  you  later  came  to  do  more  easily  or  something 
you  now  take  for  granted;  but,  for  some  reason,  your  first  time  was 
memorable.    Imagine  that  you  ar ^  writing  to  someone  who  has  just  had 
a  similar  experience,  OR  to  so^-eone  who  is  abcut  tc  have    suc^^  an  exps-^^ e-:e . 
Your  writing  might  help  prepare  your  reader  for  t^"-e  experience,  or  it  ng^t 
he'^p  your  reader  understand  that  other  people  have  gone  throjc^  the  sa-e 
kind  of  th-ing. 

You  may  write  an  essay,  a  short  Story,  a  letter,  or  a  journal  e-try  as 
a  way  of  re-telling  your  experience. 


APPEND! >  B:  QUESTIONNAIRE 


Writing  Test  Questionnaire. 

Please  read  and  compare  the  following  three  versions  of  t^e  writlrg 
Idea  you  used  recently  for  a  writing  test. 
Then  answer  the  questions  which  follow. 


A.  Think  of  an  occasion  when  you  experienced  so-^ething  for  t^e  FI^S'  T'l . 
It  may  be  something  you  later  came  to  do  easily  or  sonething  you  now  td^e 
for  granted;  but,  for  some  reason  the  first  tir^e  was  mcT^orable.  Write 
about  this  "first  time"  so  that  your  reader  u^iderstands  how  you  felt  a-^t 
why  f^ls  experience  stayed  in  your  memory. 

You  may  write  an  essay,  a  short  story,  a  letter,  or  a  journal  as  a  way 
of  re-telling  your  experience. 

B.  Think  of  an  occasion  when  you  experienced  so-^et^iing  for  the  FIRS"  ^1*1. 
It  may  be  something  you  later  came  to  do  easily  or  so"*ething  yoj  now  ta^^e 
for  granted;  but,,  for  some  reason,  the  first  tine  was  me'norable.  I'~aci'^e 
you  are  writing  to  someone  who  has  Just  had  a  sirilar  experience,  or  tc 
someone  who  Is  about  to  have  such  an  experience.    Your  writing  rig^-t  he"? 
prepare  your  reader  for  the  experience,  or  help  your  reader  unde'^stani  fat 
other  people  have  gone  through  the  sare  kind  o"  -'^ing. 

You  may  write  an  essay,  a  short  story,  a  letter,  or  a  jo-rna^  is  a  v^e/ 
re-telling  your  ex'^erience. 

C.  There  is  a  popular  book  called  MY  FIRS'*  tj^'E,  ed-:ted  by  Be'^'^ett  Ce^', 
which  Is  a  collection  of  pieces  by  famous  pecple  who  are  describing  expe^^e*-: 
they  remember  when  they  were  doing  something  for  the  first  tire.    They  tell 
aboi>t  first  kisses,  first  time  on  stage,  first  crime,  first  tine  they  saw 
someone  famous  they  would  later  work  with,  first  political  ca"paign,  first 
time  they  were  afraid,  etc.    Some  of  their  experiences  are  unusual,  so-£ 

are  common  but  are  well-told. 

I  would  like  to  collect  a  book  of  FIRST  TIME  experiences  of  high  school 
students.    I  have  asked  a  number  of  teachers  to  allow  their  students  to  write 
about  their  FIRST  TIMES  during  a  30  minute  writing  test,  fron  which  I  will 
select  promising  pieces.    Later  I  will  contact  the  authors  of  the  selected 
pieces  and  give  them  an  opportunity  to  revise  and  polish  before  pubUcatic^. 
Chosen  pieces  will  be  copyrighted  and  the  authors  will  receive  S50  for  their 
contribution.    No  piece  will  be  used  without  the  aut^^or's  per'-ission. 

You  may  write  an  essay,  a  short  story,  a  letter,  or  a  journal  entry  as  5 
way  of  re-telling  your  experience. 
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Page  2 

Writing  Questionnaire 

I.  (cont.)  Please  answer  the  following  questions  abs  *  the  three  versions 
of  your  writing  topic  given  above: 

1.  I  wrote  on  topic   ABC  don't  r^ne'nber 

2.  I  would  prtftr  to  write  o^  :     A  B  C  r.  prefe'^e^ce 

P'ease  explain  or  connent  if  you  have  a  preference: 

 (co^^tinje  0^  t^e  reverse  sid e 

II    Circle  appropriate  nunfaer  in  the  following  responses:) 

1  »  Stronqly  disagree     2  «  Disagree    3  »  Agree    4  «  Strongly  agree 

3.  I  did  not  have  e^o^'gh  tine  to  write  1         :         3  ^ 

4.  I  had  a  hard  tine  choosing  wMch 
one  of  several  possible  experiences 

I  shouM  write  about.  12         3  4 

5.  I  had  a  hard  time  thinking  of  ANV 

experience  worth  writing  abcjt.  12  2^ 

€.    I  spent  a  lot  of  time  (at  least 
five  or  ten  minutes)  tn'Ti'ing 
about  what  I  wanted  to  write 

before  I  started.  1         2         3  ^ 

7.  Because  I  spent  so  much  tine 
planning,  I  felt  rushed  when  I 

actually  started  writing.  12         3  4 

8.  I  did  not  have  time  to  re-read 

or  proof-read  my  paper.  12         3  4 

9.  Although  I  finished  befc^e  the 
time  was  up,  I  did  not  ^e-read  ny 

paper.  3         2         3  4 

IZ.  I  believe  I  would  have  written 
very  differently  about  this  topic 
If  I  could  have  taken  the  assign- 
iT'ent  home  to  complete,  or  if  I 
had  several  days  to  work  on  it 

class.  12         3  4 

II.  I  believe  I  could  have  done  a 
better  job  with  this  assign*«ent 

if  I  had  been  given  an  hour  to  write.  12         3  4 
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Page  3 

Writing  Questionnaire 


12.  I  was  anxious  while  I  wes  writing 
because  this  was  a  timed  writing 

test  which  would  affect  my  grade.  1 

13.  I  think  i  wrote  better  for 
this  assignment  than  1 

usually  write.  1 

14.  I  enjoyed  this  assignnent 

more  than  I  usually  enjoy  writing.  1 

15.  I  would  like  to  write  about  this 

topic  when  I  am  not  being  tested.  1 

16.  I  have  written  about  this  topic 
(a  first  tine  experience)  before, 

as  a  school  assignne'^t,  1 

17.  The  particular  experience  I  chose 
to  write  about  is  one  1  have  talked 

about  or  told  someone  about  before.  1 


III.    Please  continue  circling  the  resoof^se  t*"2t  cc~es  clc 
•^escribing  your  feelings  about  yoj^SQ^'^: 

15.  I  co'^siwer  ryself  a  good  writer.  1  Z 

19.  I  p- efer  writing  acade-^ic  essays  or  no^- 
-fiction  reports  rather  than  short  s*o^tes 

or  personal  essays  about  my  own  cxpe^'iences  .  1  2 

20.  I  keep  a  diary  or  journal  of  my 

experiences  or  thoughts.  1  2 

21.  I  sometimes  write  stories  or 
poe'^s  or  other  pieces  outside  of 
school,  even  though  the  writing  is 

not  assigned .  1  2 

22.  I  have  not  had  much  experience 

with  timed  writing  tests.  1  2 

23.  The  timed  writing  test  we  took 
for  this  experinent  was  different 
from  other  timed  writing  tests  I 

have  had.  '  1  ^ 


{Please  explain  your  answer  to  23  if  your  agreed  or  strong^  aqree-; 
Continue  on  the  reverse  siae  or  this  paper.) 
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Page  4 

Writing  Questionnaire 

24.  I  think  the  timed  writing  test  we  1  2  3  4    '  j 
took  is  I  good  measure  of  my  ibilities 

as  I  writer. 

25.  I  think  the  test  we  took  requires  12  3^ 
the  kind  of  writing  I  must  be  ab'e 

to  do  for  other  high  school  English 
assignments . 
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C  I-i  a  p  t  e  r   2  ;      Part  II 

C'jmparison  and  analysis  of  rater  responses 
to  the  anchor  papers  in  the  writing 
prompt  variation  study'-^ 

Catharine   Keech   and   Mary   Ellen  McNelly 


OVERVIEW  OF  THE  STUDY 


Part   II  of  the  the  writing  prompt  variation  study 
examines  task  interpretations  of  student  writers  as 
revealed  through  their  discussion  of  the   qualities  of 
the  anchor  papers  during  a  training  session  to  prepare 
theTS  for  holistic  scoring  of  their  own  papers*  Com- 
parisons are  made  of  the  ways  the  student  interpreta- 
tions and  evaluative  criteria  differed  from  those  of 
the  expert  raters  who  also  scored  the  papers. 

In  an  a ttempt^ iden t i f y  factors  in  writing  test 
design  which  may  contribute  to  the  instability  of  stu- 
dent performance  on  direct  measures  of  writings  the 
following  report  compares  student  and  teacher  evalua- 
tive criteria  in  scoring  student  essays*     One  of  the 
many  variables  which  must  affect  test  reliability  is 
the  unpredicted  student  interpretation  of  the  writing 
task.     It   is  sometimes  assumed  by  testers  and  teachers 
that  students  are  attempting  what  they  ought  to  be 
attempting  in  response  to  a  given  writing  prompt. 
Their  writing  problems  are  thought  to  coae  from  failing 
to  achieve  what  they  set  out  to  do.     In   this  study  the 
assumption  is  made  that  sometimes  students  set  out  to 
write  something  which  is  different,  more  difficult,  or 
less  appropriate  than  what  the  test  maker  expected  them 
to  do.     Thus,   the  finished  essay,   though  successful  by 
the  student's  erlteria.   is  Judged  harshly  because  it 
does  not  fit  the  raterM  conception  of  the  task.  This 
lack  of  congruence  in  task  interpretation  between  stu-» 
dent  writers  and  raters  may  be  •  more  likely  explana- 
tion than  the  assumption  that  students  set  out  deli- 
berately to  write  ''off  topic."   Recent  research  on  the 
writing  processes  of  young  children  (Scardamalia  and 
Bereiter,    1979)   and  of  adults   (Flower  and  Hayes.  1981) 
underscores  the   importance  of  taking  into  account  the 
writer's  interpretation  of  the  given  task,  a  construc- 
tion that  necessarily  includes  self-determined  criteria 
for  successful  completion. 

"•"TKrinrepoFFTs  based  upon  research  supporcei  by  the 
National  Institute  of  Education  under  Grant  Ho.  NIE  G« 
3a«>-0034  to  the  Bay  Area  Writing  Project.  University  of 
California.   Berkeley.   Leo  Ruth.   Project  Director. 
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This  study  then  seeks  information  about  student 
interpretations  of  writing  prompts  and  evaluative  cri- 
teria through  a  potentially  rich  and  generally  untapped 
data  source:   the  peer-rating  and  the  training  session 
for  holistic  scoring,  especially  when  that  training 
session  allows  student  raters  opportunity  for   full  dis- 
cussion of  their  criteria  evolved   in  racking  the  sample 
essays  in  the  anchor  set.     By  looking  at   the  evalua- 
tions and  rankings  of  student  writers  when  they  under- 
take the  role  of  raters,  we  learn  about  their   sense  of 
task  demands  and   the  values  or  criteria  they  hold  most 
important  in  deciding  when  the  task  has  been  well- 
completed.     By  comparing  these  student  rater  evalua- 
tions to  those  of  expert  teacher  raters  who  have 
evaluated   the  same  student  essays,   the  study  uncovers 
several  sources  of  variation  in  task  construction, 
evaluative  criteria,   and  the  scoring  process  itself 
which  may  affect  student  scores  in  uncontrolled  or 
unpredictable  ways. 

This  study  addresses  two  primary  questions,  each 
©f  which  includes  a  subsidiary  question: 

1,       Do  writers  and  raters  have   the  same  perception  of 
what  the  task  is? 

la.     Are  some  task  interpretations  likely  to  generate 
weaker  writing  that  is  generally  less  competitive 
in  a  holistic  scoring  than  writing  from  other 
inter  pre  ta tions? 

2o       Do  writers  and  raters  have  the  same  perception  of 
what  constitutes  a  successful  completion  of  the 
task? 

2ao     Will  agreement  between  writers  and   raters,  on  (a) 
what         the   task,   and   (b)   what  i»  a   good  solution, 
be  better  for  some  kinds  of  tasks  than  for  others? 

(Specifically:     will  Version  B.  which  speci- 
fies audience  and  purpose  in  a  manner  likely 
to  elicit  writing  for  peers ,  be  more  likely 
to  create  conflicts  in   interpretations  and 
evaluative  criteria  than  Version  A.  which 
elicits  wr i  ting  for  the  teacher  with  an 
unspecif ied--but  implicit  rhetorical  pur- 
pose?) 


Following  the  report,   Appendix  A  provides  a 
detailed  discussion  of  each  sample  paper  used  as  an 
anchor  during  the  training  of  the  three  groups  of 
raters.     The  same  set  of  anchors  selected  by  the 
researcher/trainer,   a   total  of  1M  papers,  was  used  for 


training  all  groups.     Tne  appendix  presents  the 
researcher/trainer's  response   to  each  paper,  the 
responses  of  student  raters,   the  responses  of  novice 
teacher  raters,  and  the  responses  of  expert  raters. 
The  student  responses  were  recorded  and  reported 
Mary  Ellen  McNelly,   the  Bay  Area  Writing  Project  ' 
teacher-consultant  and  co-researcher  whose  students 
participated   in  the  study  as'  writers  and  scorers. 
Responses  of  th.e   15  teacher  novices  and  the  M  expert 
scorers  are  reported  by  the  researcher/  trainer,  Catha- 
rine Keech.     Responses  of  the  teacher  novices  (student 
teachers  in  the   1931   Writing  Credential  Prograa  at  the 
University  of  California,   Berkeley)   are  included  for 
interest  as  representative  of  the  assessment  values  and 
expectations  of  beginning  teachers.   The  expert  scorers 
are  Bay  Area  Writing  Project  teacher  consultants  with 
long  experience  in  holistic  assessment. 
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PROCEDURES  OF  THE  STUDY 


Because  the  basic  objectives  of  this  whole  prompt 
variation  study  have  been  s^t  forth  In  the  Overview 
under   Part  I,   here  the  purpose  will  be  restated  only 
briefly:   To  investigate  effects  of  variation  in  the 
specification  of  audience  on  a  timed-writing  test,  114 
high  school  students,  grades   10-12,   from  four  classes 
taught  by  the   same  teacher,*  were  given  a   thirty -a inute 
test  asicing  them  to  write  about  a  particular  personal 
experience,   an  occasion  when  the  writer  experienced 
something  for  the  first  time.     Two  versions  of  the 
prompt  were  randomly  distributed  to  students  in  all 
classes  to  provide  for  comparison  of  different  task 
constraints  and  their  effects  on  performance.     The  two 
versions  of  the  prompt,   the  student  participants,  and 
the  conditions  of  test  administration  are  discussed  in 
Part   I  of  the  study. »» 

Data  Collection  and  Analysis 

After  collecting  the  student  essays  and  a  follow- 
up  questionnaire  that  probed  attitudes  toward  the 
test- writing  experience,   the  researcher  trainer* 
returned   to  the  classrooms  and  trained  the  same  groups 
of  student  writers  to  participate  in  a  holistic  scoring 
of  their  essays,   using  the  Educational  Testing  Service 
model   for  holistic   scoring,   as  adapted  by  the  Bay  Area 
Writing  Project.     The  essays  were  coded  and  scrambled 
in  such  a  way  that  students  were  unlikely  to  be  able  to 
identify  the  writer  of  any  *ssay.  / 

The  holis.tic  scoring  process  was  repeated  with  two 
groups  of  teacher  raters,  referred  to  in  the  report  as 
"novice  teacher  raters"  or   "novices"   (teacher  trainees) 
and  "expert  teacher  raters"   or  "experts"  (experienced 
teacher   raters).     Five  comparisons  were  made  based  on 
observations  of  the  training  and  scoring  process  and  on 
the  actual  scores  assigned  by  the  three  groups  of 
raters  • 

Four  statistical  comparisons  are  reported  first: 

1.       I^n ter-ra ter  reliability  estimates .     Groups  are 

compa r ed  for  within-group  agreement,  or  interrater 
reliability  during  acoring. 


•Mary  Ellen  McNelly  is   the  teacher/researcher,  or  "The 
teacher;"  Catharine  Keech  is  the  r e s ear c he r/ t r a i n er . 
••Tne     test  instrument  and   instructions  to  students  ap- 
pear  in  Appendix  A  of  Part  I* 
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2*       Agreeme^nt  among  groups ,     Scores  given  to  indivi- 
dual  papers  by  all  three  groups  of  raters  are 
correlated  as  a  aeaAure  of  b e t wee n-gr ou p  agreement 
in  rating  the  essays, 

3»       Differences  in  scoring  parameters  aoovj  rater 

groups ,     Means,   standard  deviations,  and  frequency 
distributions  of  the  scores  given  by  each  group  of 
raters  are  compared  as  indices  of  each  group's 
general  impression  of  the  whole  sample  of  essays 
as  well  as  each  group's  use  of  different  points  in 
the  scoring  range.  ^ 

^ •       Differences   in  score  distributions'  for   three  rater 
groups .     Differences  in  the  groups'   responses  to 
the  two  different  prompt  versions  are  reported,  as 
measured  by  mean   scores  and   standard  deviations  of 
score  s  assi  gn  ed . 

Finally,   a  qualitative  comparison  of  the  rater 
differences  in  responses  to  the  anchor  papers  during 
the  training  sessions  is  made.     This  comparative 
description  which  allows  further   interpretation  of  the 
statistical  findings  suggests  differences  In  writer  and 
rater  attitudes  which  may  not  be  clearly  reflected  in 
the  statistical  data. 

Raters  As  Subjects 

It   should  be  noted  that  in  this  second  part  of  the 
study,   the  research  subjects  are  not  the  students-as- 
writers,   (as  they  were  in  the  first  part);  rather,  the 
subjects  now  are  the  studen ts-as-rater s  and   two  other 
groups  of  raters  who  aLl  evaluate  the  student  writing 
on  the  two  "first  time"  prompt  variations.     Tnere  are 
then  three  distinct  groups  of  raters  as  subjects  in  the 
study,   in  addition  to  the  researcher  who  also  beco-nes  a 
subject  of  analysis  in  her  role  as  trainer  of  all 
groups  of  raters  for   the  holistic  scorings. 

!•     Student  Raters.     The   students  involved  in 
the  holistic  scoring  of  essays  were  drawn  from  the  four 
classes  who  were  administered  the  writing  test.  (The 
make-up  of  this  population  is  fully  described   in  Part  I 
of  this  study.)   It  srhould  be  noted  that  only  three  of 
the   four  classes  tested  were  used  for   scoring  the 
papers,   and  not  every  student  within  each  class  made  a 
commitment  to  a  particular  score  during  the  initial 
tallying  of  responses.     The  sophomore  non-accelerated 
class  was  excluded  from  scoring  because  it  was  con- 
sidered to  be  potentially  less  accurate  and  consistent. 
In  addition,   the  number  of  student  responses  as 
recorded  below  varies  from  27  to  84.     The  low  counts 
are  derived  from  the   second  scoring  session  when  only 
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one  class  was   retrained  using  the  remaining  anchors. 
The  high  counts  were  taken  in  the  first  scoring  sss- 
sions,   held  in  all  three  classes.     Not  all  students 
voted  on  every  paper  during  training. 

2  ,     No  vice  Teacher-Raters 

In   a  second  scoring  of  the  essays,  teacher 
trainees  in   the  Writing  Teacher  Credential  Program  at 
the  University  of  California  at  Berkeley  were  taught 
holistic  scoring  using  the  student  essays  produced  for 
this  study.     As  with  the  student  raters,   the  numbers  of 
responses  for  each  anchor  paper  vary.  Twenty-three 
teacher   trainees  participated  in   the  first  tallying  of 
responses  to  Saaples  A,   B,   C,   H,   I,   J.   and  N,   with  only 
15  teacher  trainees  participating  in  the  second  tally- 
ing using  the  remaining  saaples.     These   15  novice 
teacher  raters  also  did   the  lotual  scoring  of  the  stu- 
dent essays. 

I*     Expert  Teacher  Raters 

Four  Bay  Area  Writing  Project   teacher  consultants 
were  h.ired.     All  had  long  experience  in  holistic  scor- 
ing  for  their  own  school  districts  as  well  as  for  the 
Educational   Testing  Service.     All   had   scored   the  Cali- 
fornia High  School   Proficiency  exaaination,   and  several 
had  read  the  Advanced  Placement  exauination  in  English 
and/or   the  English  Composition  Test  given  as  part  of 
the  College   Board  examination.     Designated  as  expert 
raters,   all  four  had  taught  high  school  English  for  20 
years  or  more  each,   to  students  similar  to  the  those  in 
the  sample . 

Anchor  Papers 

The  "anchor  papers"   are  selected  in  order   to  allow 
the  establishment  of  scoring  categories  that  can  be 
easily  distinguishable  by  raters.     They  represent 
papers  which  combine  both  strengths  and  weaknesses  in 
ways   typical  of  other  papers  in  the  sample.   Thus,  exa- 
Mining  and  discussing   these  anchor   papers  provides 
readers  with  models  for  making  accurate  and  reliable 
rapid   impression  decisions  while  assigning  scores  to 
the  remaining  papers. 

The  trainer  works  within  the  limits  of  the  scoring 
points  allowed:   for   this  study,   a  range  of  1-U  was 
used,   with   1   representing  the  poorest  papers  and  the 
best  papers  in  the  set. 

Typically*  a   trainer  of  holistic  raters   (wno  may  be 
assisted  by  table  leaders)   reads  from  10  to  20  percent 
of  all  the  essays  in  order  to  select  from  10  to  30 
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sample  papers  which  can  be  used  to  "anchor"  raters  dur- 
ing  training.     CoamDn  experience  suggests  that  the 
larger   the  number  of  papers  read  before  choosing 
anchors,  the  greater  the^  likel ihood  of  representing  in 
accurate  proportions,   the  entire  range  of  writing  qual- 
ity in  the  total  sa-nple,  and  the  less  the  likelihood  of 
selecting  a  paper  which  is  atypically  high  or  low. 
Such  a  paper  would  onake  a  poor   anchor  because  it  would 
establisn  too  narrow  a  range  of  criteria  for  a  particu- 
lar  scoring  point. 

In  the  current  study,   the  researcher/ trainer  read 
the  entire  sample  of  114  essays  before  selecting 
anchors  in  order   to  gain  a  sense  of  the  range  of  qual- 
ity as  well  as  i  sense  of  what  would  constitute  uppers- 
half  versus  lower-half  papers.     Besides  including  best 
and  worst  essays   and  a   representative  selection  of  aid-- 
die  range  papers,  the  trainer  selected  one  paper  which 
seemed  to  constitute  an  evasion,   or  refusal   to  write 
about   the  assigned   prompt  (Sample  M,   discussed  below), 
and   one  unfinished   paper  which  broke  off  in  aid-word 
(sample  F,   below).     T>^ese  latter  two  provided  examples 
of  typical  scoring  problems.   Papers  representative  of 
responses  to  both  prompt  versions  A  and  3  were  also 
chosen.     In  addition,  papers  in  several  forms,  such  as 
diary  and  letter,   as  well  as   informal   essays   (the  most 
consmon   form  used),  were  chosen  to  allow  raters  to  dis«. 
cuss  any  scoring   problems  that  might  arise  from 
attempting  to  compare  quality  of  writing  across  genras. 

Training  the  haterji^ 

In  training  the  student  raters  and  the  novice 
teacher  raters  to  rank  order  a  set  of  sample  papers,  or 
anchors,  the  researcher/trainer  observed  which  papers 
the  raters  found  superior  and  elicited   from  them  the 
reasons  for   their  choices.     Tnus,   their  implicit  stan- 
dards for  writing  this  particular  kind  of  paper,  theJr 
interpretation  of  the  writing  task,  as  well  as  other 
constraints  surfaced  as  they  evaluated  and  ranked  the 
anchor  set» 

Training  the  Studejr^t  Raters 

After  a  brief  introduction  to  the  nature  and  pur- 
pose of  holistic  scoring,  sets  of  anchor  papers  were 
distributed  to  all  students  in  one  class  period.  Stu- 
dents were  asked   to  read  quickly  through  anchors  A,  B, 
C,   H,    I,   J,   and  N,   forming  an   impression  of  the  range 
of  quality  in  this  set.     They  were  told  to   choose  the 
paper   they  thought  most  outstanding  and  give  it  a  score 
of  U,   after  which  they  should  choose  the  poorest  paper 
and   give  it  a  score  of   1.   They  were  told  that  there 
might  be  more  than  one  paper  they  thought  was  best,  in 
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iihloh  case  they  could   give   two  scores  of„4.     In  the 
saaxe  way,   tney  ware  allowed  to  give  two  scores  of  1. 
After  establishing  the  outer  boundaries  of  the  scoring 
range,   they  were  directed  to  rank  order   the  remaining 
papers  until  all  papers  had  scores  and  all  scores  had 
been  used  zt  least  once. 

Tne  student  raters  were  trained  during  their  regu- 
lar English  class  periods.     The   training  sessions  were 
consistent  in  all  three  of  the  upper-level  classes  used 
for   the  scoring.     The  votes  generated   for  a  particular 
paper   fell   into   similar  patterns  in  all  classes.  It 
proved  possible   for  the  researcher/trainer  to  maintain 
a  consistent  scoring  categories  across  all  of  the  class 
per  iod  s  r 

As  students  read  the  anchors,  the 
researcher/trainer  placed  a  grid  on  the  board  as  shown 
in  Figure   1.*  Tnen  when  they  were  ready,   the  researcher 
called   for  a  show  of  hands  to  indicate  which  scores 
students  had  assigned  each  paper.     The   tally  revealed 
that  the  majority  of  students  ranked  Example  A  exactly 
as  the  researcher-trainer  had,   although  some  students 
had   ranked  it  higher  or  lower,   as  is  usually  the  case 
in  the  early  stages  of  training   inexperienced  raters. 
Discussion  of  exaaiple  A  followed  in  which  low  raters 
eited  its  weak  points,  while  high  raters  defended  their 
higher  ranking.     This  discussion  is  the  heart  of  the 
training  process,   allowing  raters  to  become  sensitive 
to  qualities  and   flaws  that  they  had  not  previously 
attended  to,  as  well  as  helping  them  to  adjust  the 
relative  importance  they  attach  to  different  strengths 
and  weaknesses  in  the  light  of  how  successful  the  piece 
of  writing  is  on  the  whole,   compared  to  other  writing 
in  the   total   sample.     They  begin   to  develop  a  sense  of 
what  a  2  versus  a  3  versus  a  4  might  mean   for   this  sam- 
ple of  papers.   It  is  also  during  this  process  that  the 
students  revealed  what  they  valued  in  a  piece  of  writ- 
ing:  what  their  criteria   for  good  writing  were  and  how 
they  had  construed  the  task  to  begin  with.  (Their 
evaluative  responses  to  each  anchor  paper  are  summar- 
ized  in  Appendix  A,   along  with  the  responses  of  the 
teacher  raters.) 

As  training  progressed  within  each  class  period, 
student  raters  in  each  class  began  to  agree  more 
closely  on   their  rankings  of  papers  whicn  were  clearly 
representative  of  a   particular  scoring  point.     One  or 
twopapars,  however,   were  awarded  a  broad  spread  of 
scores;   these  would   typically  cause  discrepant  scoring 
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•All  figures  and  tables  are  placed  at  the  end  of  the 
report . 
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if  encountered  during  the  reading.     As  useful  as  sus- 
tained analysis  of  such  papers   is,   there   is  also  a 
danger   in  analyzing  anchors  too  closely.  Over-extended 
discussion  of  specific  features  of  a  paper  may  mislead 
raters,   costing  thea  their  ability  to  form  a  whole 
impression  of  a  work  and  making  it  difficult  to  compare 
it  to  other  papers  for   the  overall  effectiveness  of  its 
unique  combination  of  flaws  and  strengths. 

In  the  training  of  the  students  raters,  as   in  the 
training  of  any  group  of  inexperienced  readers,  the 
rather  open  discussions  of  each  paper  were  concluded 
with  a  strong  summary  of   its  general  worth  to  confirm 
the  majority  vote  and   to   clearly  desifnate  the  essay  as 
belonging  to  one  scoring  category  or  another.  Whenever 
an  essay  appeared   to  fall  between  categories,   a  split 
vote  was  sometimes  allowed  to   stand,   since  prolonged 
discussion  or   arbitrary  rulings  often  serve  to  make  the 
raters  insecure  about  their  category  boundaries.  In 
such  cases,   students  were  reminded  that  each  paper 
always  receives   two  scores,   and  that  the  combined  rat- 
ings on  such  a  paper  would  be  likely  to  reflect  the  two 
views,   yielding  a  final   score   (of,   for  Instance,  2-^3 
s  5)   which  would  be  descriptive  and  fair. 

Training  the  Hovice  Teaciher  Raters 

The  training  of  an   initial  group  of  23  novice 
teacher  raters  was   part  of  a  general   workshop  on  holis- 
tic  scoring  and   followed   an  introduction  to  assessment 
theory  not  given  to  the   student  raters.     However,  in 
all  other  respects,   the   same   training  procedures 
described  above  were  used,   with  the  r e se a r c her / tr a i ne r 
eliciting  the  novice  teachers*   untutored  responses  to 
each  anchor  paper  before  attempting  to   summarize  and 
achieve  a  consensus.   This  first  group  of  23  student 
teachers  tended   to  use   the  training  session  more  as 
forum  for  exploring  and   shaping*  their  own  criteria  for 
evaluating  writing.     In   general,   classroom  experience 
had  not  yet  provided  them  the  opportunity  for  testing 
their   ideas  the  classroom.     Consequently  this  novice 
teacher  group^s  operative  criteria,   as  expressed  during 
the   training,   never  stabilized  during  the   first  train- 
ing  session  to  the  same  extent  as  the  student  raters* 
responses  had  done.     A  second   training  session  was  held 
a  week  later  with   15  teacher  novices  from  the  original 
group  who  proceeded   to   score   the  entire  sample. 

Training  the  Expert  Teacher  Raters 

The  same  set  of  anchors  used   to   train  student 

raters  and  novice  teacher  raters  was  given  to  the  four 

expert  teachers  for   ranking.     They  quickly  reached  a 

high  rate  of  agreement.     However  the  speed  and 
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confidence  with  which  they  reached  agreemant  on  the 
proper  scores  to  assign  each  anchor  papar  proved  to  be 
misleading  and   led  to  a  problea   in  defining  certain 
categories  during  the  rtading.     Even   though  the 
researcher/trainer  observed   that  the  expert  raters 
seeded   to  be  defining  the  category  of  3  too  broadly  and 
feared   that  too  few  papers  in  the  full  sauple  would 
fall   into  the   2  category,   she  honored  the  distinctions 
that  the  experts  wished  to  draw,   believing  that  letting 
them  determine  the   scoring  boundaries,   which  seeaed  so 
clear  to  then,   would   increase  the  reliability  of  their 
SCO  r  i  n  g . 

The  researcher/trainer  chose  not  to  intervene  to 
narrow  the  3  categories  also  because  she  was  aware  that 
the  larger  proportion  of  students  in   the  saaple  were 
quite  good  writers,  with  only  one  class  in   four  made  up 
of  younger,   less  experienced  writers.     She  thought  the 
experts'   classification  of  the  papers  might  more  accu- 
rately reflect  the  actual  distribution  of  ability  in 
this  skewed  group.     And  the  r e se ar ch e r/ tr a i n er  recog- 
nized  that  it   is  an   innate  problem  of  holistic  scoring 
and  other  rank-ordering  that  the  original  scale  which 
is  applied  to   a  set  of  papers  does  not  necessarily 
reflect  eqaal   distances  between   integers.     There  may  be 
Inherent  in   s-tting   the  scale  a  conflict  between  the 
practice  of  trying  to  distribute  students  in  a  fairly 
normal   distribution   across   scoring  points  and  the  prac- 
tice of  trying  to  arrive  at  scoring  points  that  are 
equal   intervals  apart  in  terras  of  the  quality  of 
essays,   rather  than   in  terras  of  the  number  or  percen- 
tage of  essays  that  fall   in  each  interval.     Having  ear- 
lier defined   the  intervals  by  trying  to  distribute  stu- 
dents across   all  scoring  categories  in  something 
approaching  a  normal  curve  (albeit  skewed  by  nature  of 
the  population) ,   the  researcher-trainer  withdrew  from 
the  discussion  of  the  experts,  who  seemed   to  be  drawing 
on  a  common  experience  and  making  reference  to  outside 
criteria  to  determine  the  scale  point  intervals  in 
terms  of  quality  of  writing. 

As  the  reading  progressed,   however,   it  became 
clear   that  the  expert,    themselves  believed   that  they 
had  defined  the  category  of  3  too  broadly  in  scoring 
anchors.     They  described  seeing  the  3  category  as  "high 
threes"  and  "low  threes" — a  common  practice   in  large 
readings,  where  a  few  scoring  points   are  used  to 
reflect  a  broad  range  of  writing  quality.     But  they 
noticed  that  other  categories  such  as  2  or   M  did  not 
get  broken  up  in  their  minds.     They  agreed  with  one  of 
the  raters  who  said  that  a   five  point  scale  was  needed 
for  these  papers  to  deal  with  their   feeling  that  too 
large  a  range  of  papers  received  a  score  of  3.  The 
papers  with  this  score  were  more  different  from  one 
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another  than  were  the  papers  within  any  other  scoring 
interval  . 


Rating  the  Papers 

Student  raters  spent  asost  of  the  first  day  reeeiv- 
ing  training  in  holistic  assessment,  using  the  anchor 
papers.     In   the  final   10  minutes  of  each  50  ainute 
class  period,   papers  were  distributed  randomly  to  the 
class,   two  to  each   student.     Essays  were  coded,  and 
essays  from  all  classes  were  included  in  the  sample 
distributed  to  any  one  class.      (Students  w^re 
instructed   to  exchange  papers  with  another  student  if 
they  had  their  own  or  thought  they  recognized  the 
writer  of  the  paper  they  were  scoring.)   Students  were 
informed  that  the   teacher  and  the  researcher/trainer 
would  be  spot  checking  the  holistic  scores  assigned  in 
the   interest  of  fairness  to  the  writers,   but  that  in 
most  cases,   the  total  score  assigned  by  the  student 
raters  would   constitute  the  writer's  score  on  the 
essay.     Students  were  again  reminded  that  their  rating 
was  only  one  of  two  the  paper  would  receive. 

The  scores  given  on  the  first  reading  were  con« 
coaled  before  the   student  raters  passed  the  papers 
along  for   the  second  reading. 

On  the  second  rating  day,    students  in  the  two 
largest  classes   (classes  which  had  shown  the  greatest 
stability  during  training)   completed  scoring  the  essay 
set  after  being  retrained  with  the  remaining  anchors, 
samples  D,   E,   F,   G,   K,   L,   and  M.   Results  of  their  dis- 
cussion of  the  latter  anchor  set  are  also  included  in 
the  appendix.     The  teacher  and   the  researcher/trainer 
re-scored  one  third  of  the  essays  and  ^ach  found  only 
one  paper   which  was   inaccurately  scored  by  both 
readers.     Interrater  reliability  estimates  arc  reported 
in  Table   1 . * 

Novice  teacher  raters  began  and  completed  scoring 
in  one  two-hour  session,   after  being  retrained  with  the 
same  set  of  anchors  used  in  the  student  retraining. 
Tne   researcher/trainer   acted  as  table  leader,  moving 
around  the  room  to  check  the  ratings  of  each  rater  at 
least  twice  during  the  two  hourSc     Four  tables  of 
readers  drew  from  randomized  piles  of  essays   in  the 
center  of  the   tables  and  concealed  the  first  ratings 
from  the  second  raters.     Papers   scored  at  one  table 
were  distributed  to  another   for  the  second  ratings. 
After  the  scoring  had  ended,   all  dlscrepantly  scored 
papers  were  given   a  third  reading  and  a  new  total 


•Ail   tables  appear   at  the  end   of  this  report 
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score.     Inter-rater  reliability  estimates  for  this 
group   are  reported   in   Table  1. 

The  four  expert  teacher  raters  readers  sat  with 
the  researcher/trainer  around  a   single  table  for  the 
training  and   scoring  of  the  sauple.     Papers  were  ran- 
doaized   and  divided   into  eight  stacks.  Raters 
exchanged   papers  in  a  rather  co.-nplex  arrangement 
intended  to  insure   that  every  rater  was  matched  to 
every  other  rater  for  roughly  equivalent  proportions  of 
the  reading.     All  discrepancies  we re  resolved  through 
provision  of  a  third  reading  during  the  same  rating 
period.     Inter-rater  reliability  estimates  for  this 
group  are  reported   in  Table  1. 
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STATI3TTCAL  COMPARISON   OF  RESULTS 
FROM  THREE  GROUPS  OF  HATERS 

There  are  a  nuubsr  of  ways  to  describe  the  aTJOunt 
of  agreement  aiong  raters,   both  within  the  three  rating 
groups,   and  axong  the  groups  In  a  comparison  of  the 
cnree   ratings  of  the  saije  set  of  student  essays,  Tne 
first  two  statistics  of  Importance  .ueasure  wlthln-group 
rater  agraeuent,   or  what  Is  usually  known  as  Inter- 
rater  r el  lab  lltty . 

Inter -rater  Reliability  Estimates 

There  are  two  accepted  ways  of  describing  the 
anount  of  agree nent  between  raters  of  writing  sauples 
in  holistic  assessment.     The  first   Involves  simply 
reporting  the  number  or  percentage  of  essays  which 
received   scores  from  two  raters  with  more  than  a  one 
point  difference.     rflth  each  rater  using  a  scale  of  1 
to   4,   so-called  discrepant  ratings  would   include,  for 
example,   a  rating  of  2  from  the   first  rater   and  a  rat- 
ing of  4  from  the  second  rater,   or   paired  ratings  of  1 
and  3t   or    1   and  4. 

Reported   in  the  final  column  of  Table   1   are  the 
discrepancy  percentages ,   or  percentages  of  total  essays 
wnich  received  discrepant  ratings  from  two  readers, 
entered  as  d   =   .078,   for  example,   or  7.81.     As  training 
procedures  have  improved   and  the  pool  of  experienced 
readers  has   increased,   local  holistic  assessments  in 
California   typically  report  discrepancy  percentages  of 
less  than    101,  many  regularly  achieving  less   than  5X, 
and  some  places,   such  as   San  Francisco   State  Univer- 
sity, which  maintains  similar  criteria  from  year  to 
year,   report  consistently  achieving  fewer   than  1X 
discrepancies. 

rfhen   the  students  who  wrote  the  essays  assumed  the 
role  of  raters  and  were  trained   in   holistic  scoring, 
they  achieved   a  respectable  7*8t  discrepancy  rate, 
while  novice  teacher  raters,  who  proved   less  responsive 
to  training,  produced  a   151  discrepancy  rate.  As 
expected,    the  experienced   teacher   raters  maintained  a 
low,    (4. MX),  discrepancy  rate. 

A  second  estimate  of  inter-rater  reliability  is 
usually  reported  as   the  degree  of  association  or  the 
correlation  between  the  scores  of  the  first  and  second 
raters  on  all  papers.     Tne  measure   used  here  is 
Pearson's  r,  with  perfect  correlation  represented  by 
1,00   (all   first  and  second  readers  agreeing  exactly  on 
tne  scores  assigned  to  each  paper),   and  no  correlation 
represented  as   .000   (no   first  and  second  readers  agree- 
ing exactly  on  the  score  of  any  paper,  with  both 
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maximuji  amount  and  randoai  direction  of  disagreement; 
however,   this  is  a  theoretical,   not  a  practical,  possi- 
bility).    Values   for   Pearson's  r  of   .01   to    .35  are  con- 
sidered quite  low,   for  most  purposes;  correlations 
better  than  r  s   .35  up  to  but  less  than   .65  are  gen- 
erally considered  moderate  rates  of  agreement,  while 
correlations  of  more  than   .65  are  considered  fairly 
strong  rates  of  agreement.     In  the  early  days  of  holts- 
tic   assessment,   the  best  rates  of  agreement  between  two 
readers  were  rarely  over   .60   (Diederich,    197^).  More 
recently,   testing  programs,   including  not  only  the  pro- 
fessionals  at  Educational  Testing  Service  but  many 
local  schools,   have  been  reporting  raw,  unadjusted 
correlations  between  readers  of  .7,    .8,  and  in  some 
instances  of  better   than  .9,* 

Table    1   reports   the  correlations  or  inter-rater 
reliability  estimates   for   the  three  groups  of  raters, 
showing  the  student  raters  achieving  a  statistically 
significant  correlation  of  r  s   .68   (p  *   .01)  which 
approaches  a  generally  acceptable  rate  of  agreement  by 
today's  standards.     The  novice  teacher  raters,  on  the 
other  hand,   produced  only  a  moderate  correlation  of  r  s 
.55   (p  s   .01),   which  is  considered  a  poor  rate  of 
agreement  for  raters.     The  experts  achieved  a  correla- 
tion of  r   s   .71    (P  =   .01),   as  good  a  correlation  as 
that  achieved  by  experienced  raters  in  many  local 
assessments,   and   in   some  professional  assessments,  as 
well,  but  lower   than  anticipated  by  tht?  researcher- 
trainer  who  expected   the  four  experts  chosen  to  be 
highly  compatible  readers.     The  discussion  below  con- 
siders possible  reasons  for   this  less  than  outstanding 
rate  of  agree^aen  t . 

Agreement  Between  Groups 

For  comparing  scores  assigned  to  the   same  essay  by 
the   three  different  groups  of  raters,  only  the  essays' 
final  or   total  corrected  holistic  scores  were  used 
(ranging  from  2-8  possible  points).     For   instance,  an 
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•Diederich  reportsa  procedure  for  "stepping  up"  relia- 
bility estimates  by  using  two  or  more  samples  of  stu- 
dent writing.  This  cannot  be  done,  of  course,  where 
only  one  sample  of  writing  is  rated  for  each  student. 
Also,  since  discrepant  scores  are  always  resolved  by 
giving  the  paper  a  third  rating,  which  is  used  to  pro- 
vide a  "corrected**  or  adjusted  score  for  the  essay,  it 
is  possible  to  calculate  the  measure  of  association  us- 
ing these  corrected  scores.  This  is  not  generally 
done,  however,  since  testers  usually  wish  to  estimate 
the  extent  of  agreement  between  the  original  two 
raters,   to  test  the  accuracy  of  the  reading  as  a  whole. 
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essay  fflight  have  received  a  score  of  5  fronj  the 
student's  rating,   (representing  the  two  scores  of  2  ♦ 
3),  while  the  novice  teacher  raters  rewarded  the  saae 
paper  with  a  total  score  of  6  (3  ♦  3)f  and  the  expert 
teacher  raters  judged   it  even  more   favorably,   giving  it 
a  score  of  7  (3  ♦  Pearson's  r   is  used  again  to 

report  correlations  between  the  total  holistic  scores 
awarded  by  each  group* 

Table  2^  reveals  that  the  poorest  agreement  was 
between  student  raters  and  novice  teacher  raters  at  r  s 
.47   (p  s   .01).     The  best  agreement  was  between  novice 
teacher  raters  and  expert  teacher  raters,  at  r  s   .62  (p 
s   .01).     Students  agreed  with  the  expert  teacher  raters 
only  moderately,   r  s   .50  (p  s   .01).     (Novice  teacher 
raters  did  not  score  8  of  the  papers  because  duplica* 
tion  of  original  was  inadequate  for   rating.  Student 
raters  and  expert  teacher  raters  scored  the  full  sam-» 
pie,  working  with  original  papers.) 

These  simple  correlations  merely  alert  us  to  differ- 
ences between  the  groups  of  raters  in  valuing  the 
papers;  the   following  statistical  analysis,  together 
with  the  narrative  descriptions  of  rater  training  and 
responses  to  specific  essays,  provide  more  insight  into 
the  nature  and   extent  of  those  differences. 

Pi  ff erences  In  Scoring  Parameters  Between  Rater  Groups 

Table   1  reports  means,  modes,  medians,  and  stan- 
dard deviations  of  the  scores  given  to  the   full  set  of 
essays  by  each  group  of  raters,   respectively.     Thus,  we 
can  see  that.   If  the  student's  own  ratings  were  used  to 
estimate  the  writing  abilities  of  the  groups  who  wrote 
this  set  of  essays,   the  entire  group  would  earn  a  mean 
score  of  5.66,   with  a  score  of  5  (3       2)   being  the  most 
common   score  awarded   (the  mode).'   Novice  teacher  raters 
provide  a  more  positive  estimate  of  group  ability,  the 
average  of  all   scores  awarded  being  5.95«   with  the  most 
common  score  being  6.     Expert  teacher  raters  agree  with 
novice  teacher  raters  in  producing  more  ratings  of  6 
than  any  other  score,  but  reward  the  set  of  essays  with 
an  even  higher  mean  score  of  6.18. 

The  difference  between  these  means  may  not  appear 
to  be  great,   but  comparing  expert  teacher  raters  to 
student  raters   (the  original  comparison  of  interest), 
the  difference   is   .52,   about  half  a  scoring  point  (sta- 
tistically significant,  p  =   .05)   or  Just  over   1/3  of  a 
standard  deviation.     Such  a  difference  between  holistic 
scores,   earned  by  different  groups  of  writers,  is 


•Tables  appear  a  the  end  of  this  report. 


274 


usually  considered  educationally  significant, 
representing  i  greater  difference  than  usually  occurs 
when  two  groups  of  similar  ability  are  compared  after 
writing  on  the  saae  essay  topic   (prorapt),  or  when  the 
saise  group  Is  compared  before  and  after  Instruction, 
writing  either  on   the  same  or  on  different  essay  topics 
(prompts).     In  other  words,  student  evaluations  of  the 
paper s  in   this  set  are  significantly  lower  than  teacher 
evaluations  of  the  same  set* 

Differences  in  Score  Distribution  for  Three  Rater 
Groups 

Reporting  only  the  numerical  parameters  of  the 
scores  assigned  by  each  group,   however,  obscures  addl«- 
tlonal  Important  differences  in  the  scoring  patterns  of 
these  groups.     Such  differences  invite  interesting 
speculation,   althougn  they  generally  support  the  find- 
ing  that  these  students  seemed  willing  to  judge  their 
work  more  harshly  than  experienced  teacher  raters  did. 

Table  2.2  shows  the  distribution  of  scores  given 
by  the  three  rater  groups.     It   is  evident  at  a  glance 
that  the  student  raters  came  closest  to  providing  a 
normal   distribution  of  scores  of  the  sort  expected  in 
large-scale  holistic  scorings.     They  were  more  willing 
to  use  the  lower  end  of  the  scoring  range,  with  six 
papers  receiving  a  score  of  1   from  at   least  one  reader 
(total  scores  for   these  papers  would  be  either  2  or  3, 
depending  on  the  second  rating).     No vice  teacher  raters 
used   fewer  scores  of  1   (on  only  three  papers),  and  the 
expert  teacher  scorers  gave  only  two  papers  a   1,  and 
even  then,   these  papers  both  received  a  score  of  2  from 
their  second  rater,  so  that  not  one  paper  in  the  sample 
was  given  the  fflinlraua  score  of  1   ♦  1   s  2.  This 
occurred  despite  the  fact  that  the  expert  teacher 
raters  had  agreed  to  give  a   1  rating  to  one  of  the 
anchor  paper s  thereby  indicating  it  as  typical  of  the 
category  of  writing.     When  encountering  the  sarne  paper 
during  the  rating,   at  least  one  expert  teacher  rater 
changed   her  mind  and  gave  it  a  2. 

Student  raters  also  had  a  broader  sense  of  the  2 
category  of  paper,   distributing  papers  in  the  middle 
range   (scores  of  2  or  3)   equally  across  both  score 
categories,  with  the  same  number  of  papers  (23)  receiv- 
ing  two  ratings  of  2  (total   score  x  4)   and  two  ratings 
of  3  (total   score   s  6).     Slightly  more   papers  (26)  fell 
halfway  between  with  scores  of  2  ♦  3  -  5.  Novice 
teacher  raters,  on  the  other  hand,  awarded  a  score  of  2 
to  fewer   papers,   preferring  to  use  a  3  score   for  a 
broader  range  of  essays.     The  expert  teacher  raters 
carried  this  tendency  even  further  and  placed  47  papers 
in  the  3^3*5  category,  with  only   10  papers 
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receiving  scores  of  5  and  only   13  receiving  scores  of 
4.     Possible  reasons  for  this  odd  distribution  are  sug- 
gested  in  part  by  differences   in  the  training  of  the 
groups,   and  are  more  fully  explored  below. 

Discussion  of  Results 

The  differences  between  groups  in  the  inter-rater 
reliability  achieved  are  perhaps  not  surprising.  The 
students  were  systematically  trained;   the  criteria  for 
good  and  poor  essays,  after  being  elicited  from  stu- 
dents,  were  re-articulated  and  delineated  by  the 
researcher/trainer,  and  the  students  appeared  eager  to 
cooperate,   to  "learn  how  to  do  it  right."  As  a  result, 
they  achieved  a  quite  respectable  correlation  between 
raters  of  .68.  With  the  novice  teacher  raters,  on  the 
other  hand,   there  was  less  willingness  to  reach  con- 
sensus.    They  persevered  with  their   idiosyncratic  read- 
ings of  various  essays,  exactly  as  one  aight  expect  a 
group  of  inexperienced  teachers  to  perform  without 
benefit  of  moderating  influences  accumulated  through 
classroom  experience.     Tne  result  was  a   low  correlation 
between  novice  teacher  raters  of  only   .55.     The  expert 
teacher  raters  were  confident  and  compatible  in  their 
opinions,   having  done  many  scorings  together   for  the 
Educational   Testing  Service.     No  attempt  was  made  to 
moderate  their   views,   with  the  result  that  the  scores 
they  gave  to  the  anchor  papers  may  have  been  less 
discriminating  guides  than  they  might  have  been.  But 
even  so  the  expert  raters  achieved  a  consensus  which  in 
turn  resulted  in  a  reasonably  good  correlation  between 
first  and   second  readers  of  .71. 

Failure  to  achieve  a  closer  correlation   among  the 
expert  readers  may  be  attributed,  at  least  in  part,  to 
certain  factors  which  may  affect  reliability  in  any 
holistic  scoring.     The  two  versions  of  the  prompt  eli- 
cited.  In  some  cases,   very  different  content  and  modes 
of  writing,   with  the  audience-emphasis  version  evoking 
from  some  students  what  might  be  called  process  essays, 
or  directions  on  how  to  do  a  particular  thing  for  the 
first  time   (See  Sample  D,  Appendix  A).     These  latter 
essays  were  different  In  many  respects  from  the  typical 
response  to  either  version  of  the  prompt,   which  usually 
was  narrative  In  structure,  and  provided  an  anecdotal 
account  of  the  writer's  own  "first  time."  Other  work  in 
this  assessment  project   (See  Technical  Report  No.  5) 
Illuminates  the  difficulty  that  raters  have  In  compar- 
ing two  very  different  types  of  writing  In  a  single 
rank-ordering  based  on  over-all  quality.     In  general, 
as  shown   In   the  qualitative  description  of  the  training 
sessions  (See  Appendix),   the   process  papers  suffered  by 
comparison  with  the  narrative,  autobiographical  papers; 
however,   some  raters  responded  more  positively  to  the 


process  papers,  believing  that  giving  directions  is  a 
more  difficult  task  than  narrating  an  event.   Thus  they 
over-rewarded  the  writers  of  these  papers. 

Another   factor  which  may  have  contributed  to  the 
disagreement  aaong  raters  is  the  instruction  on  the 
writing  test:  "You  may  write  an  essay,   short  story, 
letter.  Journal,  etc."  It  is  difficult  to  make  fair 
comparisons  among  pieces  written  in  these  different 
genres.     In  addition,  as  can  be  seen  in  the  discussion 
of  responses  (Appendix  A)  letters  and  diaries  present 
special  problems  to  readers  who  may  choose  either  to 
forgive  or  not  to  forgive  infelicities  of  expression, 
informal  language,   casual  organization,  and  lack  of 
development  or  lack  of  vivid  concrete  details,  when 
these   features  represent  the  student's  accurate 
interpretation  of  the  tone  and  style  of  a  real  letter 
or  diary  entry. 


Some  of  the  disagreement  revealed  in  the  low 
correlation  of  scores  between  groups  and  in  the  strik- 
ing differences  in  scoring  distributions  may  be  attri- 
buted to  the  training  and  the  initial  defining  of  the 
scoring  categories.     During  training,   the  student 
raters  were  encouraged  to  consider  virtues  and  faults 
of  the  anchor  papers  identified  by  the 
resear Che r/ trainer  .     More  amenable  to  the  training, 
they  were  more  likely  to  divide  the  sample  in  a  way 
that  distributed  papers  normally  across  scoring  inter- 
vals.    On   the  other  hand,   the  expert  teacher  raters  who 
rapidly  defined  their  scoring  categories  may  have  drawn 
on  prior  definitions  of  the  scoring  points.  Possibly 
they  may  have  been  Influenced  by  the  five-point  scale 
they  were  each  familiar  with  from  scoring  the  Califor- 
nia High  School   Proficiency  Exam  (CHSPE)   and  by  the 
quality  of  essays  drawn  from  the  general  high  school 
population  that  they  typically  encounter  while  scoring 
that  test. 


More  directive  procedure  for   training  the  expert 
teacher  raters  might  have  yielded  different  results.  Or 
if  the  expert  teacher  raters  themselves  had  read  the 
entire  sample  before  pulling  anchors,  rather  than  hav- 
ing the  researcher/trainer  choose  the  anchors,   the  dis- 
tribution of  scores  might  have  more  closely  resembled 
the  students'  distribution. 
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There   is  no  question  that  the  process  of  choosing 
anchors  to  define  a   scoring  range  is  a  critical  step  in 
any  holistic  assessment.     Choosing  anchors  requires  the 
Juggling  of  wnat  may  be  contradictory  constraints.  On 
the  one  hand,   there   is  the  need  to  distribute  the  sam- 
ple quantitatively  according  to  some  cr iteria--either 
that  the  distribution  represent  the  distribution  of  the 
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population  on  other  measures  of  related  ability,  or 
that  the  population  be  equally  distributed  across  scor- 
ing  points,  or  that  the  population  be  nornally  distri- 
buted across  the  scoring  range.     On  the  other  hand  ,  the 
further  constraint— that  the  scoring  intervals  should 
represent  roughly  equivalent  differences  in  quality 
among  essays— is  also  imposed.     In  practice,  the  latter 
constraint  aay  be  defined  by  the  former:  testers  may 
come  to  feel  that  the  operational  definition  of 
"equivalent  differences  in  quality"   from  one  scoring 
point  to  another  should  be  that  "equivalent  or 
appropriate  numbers  of  essays"  are  grouped   into  each 
scoring  interval.     This  complex  interplay  of  quantita- 
tive and  qualitative  considerations,  an  interplay  which 
takes  place  subjectively  for  the  most  part  during  the 
selecting  of  anchor  papers,  deserves   further  study. 

Even   if  student  raters  and  teacher  raters  arrived 
at  similar  distributions,  with  similar  mean  scores,  the 
agreement  of  their  verdicts  on  particular  papers, 
represented  by  the  correlation  between  student  rater 
and   teacher  rater  scores,  might  stili  remain  low  for 
the   three  rating  groups  worked  within  three  distinctly 
different  frames  of  reference   for  Judging  the  essays. 
The   student  raters'   expectations  very  likely  were 
derived  frou  familiarity  with  their  teacher's  expecta- 
tions, with  their  own  and  their  classmates'   work,  and 
with  the  standards  generally  set  forth  in  their  partic- 
ular high  school. 

The  expert  teacher  raters,  none  of  whom  teach  at 
the  school  where  the  study  was  undertaken,  had  expecta- 
tions drawn   from  a  much  larger  assessment  population, 
including  not  only  their  own  schools  '  ^;!^,, 

also  the  large  number  of  students  throughout  California 
who  -rite  for   the  proficiency  txam  which  these  teachers 
have  scored  since  its  inception.     In  general,  the  stu- 
dent writers  in  this  study  produced  better  papers  than 
might  be  found   in   these  larger   school  and  state  popula- 
tions.    Consequently  by  this  standard,  the  expert 
raters  found  fewer  rtally  poor  paptrs. 

Finally,   the  reference  points  for  the  novice 
teacher  raters  became  somewhat  confused,   and  certainly 
mixed.     The  between-group  correlations  in  Table  2  show 
this  latter  group  to  be  closer   in  agreement  with  the 
expert  teacher  raters  than  they  are  among  themselves. 
This  condition  is  easy  to  understand   if  we  imagine  that 
two   novice  teacher  raters  might  produce  a  discrepant 
scoring  of  2  ♦  U  on  a  papjr  which  two  expert  teacher 
jrteJs'might  recognize  clearly  as  a   3  J  3-  ^B°t^^-^^- 
raters  would  be  closer  to  the  ex-perts  than  they  were  to 
each  other,  which  would  explain  the  pattern  of  correla- 
tions reported. 
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Additional  explanations  for  the  lac<  of  agreement 
aaong  the  three  groups  of  raters  are  best  sought   in  a 
closer  exauination  of  the  values  and  criteria  expressed 
during  the  training  sessions,   in  reaction  to  particular 
anchor  papers.     The  Append ix  includes  individual  and 
group  responses  recorded  during  training.   It  also 
reveals  the  criteria  which  seeaed  most  operative  for 
the  three  groups  of  raters  as  well  as  for  the 
researc he r/ trainer. 

One  interesting  wi thin-»gr oup  phenomenon  remains  to 
be  explained.     Why  did  the  expert  teacher  raters  agree 
to  assign  the  score  of  1   to  at  least  one  anchor,  but 
fail  during  the  actual  reading  to  give  that  same  paper 
a  score  of  1  on  both  of  its  readings?     The  most  likely 
explanation  is  that  these  teachers  were  working  with 
pr e-*establ  ished  notions  about  the  kind  of  writing  which 
should   fall  into  the  lowest  scoring  category*     In  many 
local  high  school  assessments,  as  in  the  CHSPE.  scores 
of   1  are  reserved  for   papers  with  so  little  writing  as 
to  be  nearly  unscorable,   or  with  such  severe  scribal  or 
transcription  difficulties  as  to  be  nearly  indecipher- 
able and  severely  flawed   in  spelling  and  punctuation. 
To  keep  standards  constant  from  one  year  to  the  next, 
these  criteria  are  carried  over,  even  as  students  tak- 
ing the  test  produce  fewer  and  fewci    papers  in  this 
lowest  category.     Thus,   the  expert  teacher  raters,  may 
have  held   to  an  external  criterion  for  this  lowest 
score,  even  though  it  meant  they  almost  never  used  th« 
scoring  category.     The  other  groups,   without  this  back- 
ground experience,  were  more  willing  to  follow  the 
directions  and   try  to  use  the  full  point  range.     As  is 
evident  from  anchor  paper  (B).  which  was  scored  a  1 
during  training,  evaluative  criteria  drawn  from  other 
general  readings  would  be  inappropriate  for  this  sam- 
pi  e  • 

There  is  one  more  factor  which  may  have  affected 
scoring  results  in  unaccountable  ways:   In  the  reading 
and  rating  of  the  anchor  during  the  training  session 
typed  versions  were  used.     But  during  the  actual  read- 
ing, these  same  papers  were   in  their  original  handwrit- 
ten  forms.     The  expert  teacher  raters  reported  that 
they  had  different  feelings  about  the  anchor  papers  on 
reading  them  in  their  original  handwritten  form.  Unex- 
pectedly,  they  tended  to  want  to  give  the  handwritten 
versions  higher  scores,   in  spite  of  messy  handwriting 
or  less  than^at tractive  presentation.     The  typed  ver- 
sions apparently  made  these  teachers  more  conscious  of 
slips  of  the  pen — words  omitted  or  repeated,  faulty 
capitalization  or  spelling  that  could  have  been  errors 
in  transcription  rather  'Chan  errors  of  student 
knowledge  or  performance.     The  errors  appeared  more 
glaring  when  found  on  the  printed  page,  than  in  the 
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context  of  an   ircpromptu  hastily  handwritten  essay, 
where  they  were  generally  ignored  as  being  indicative 
of  little  more  than  lack  of  time  to  proofread. 


This  is  an  interesting  finding,   in  the  light  of 
previous  work  ( Redmond  in o .   cited  'in  Diederich.   197^.  P. 
9)  which  demonstrated   that  handwriting  is  one  of  the 
factors  affecting  teachers*   judgments  of  an  essay,  with 
poor  handwritings  lowering  scores,   and  good,  clear 
nandwriting  positively  influencing  the  reader.     In  this 
study,  however,   a  clear  handwriting  might  well  have  the 
effect  of  making  errors  more  visible.     It  is  possible 
that  the  student  who  blurs  letters  in  order   to  render 
possible  misspelling  less  obvious  may  be  using  an 
effective  strategy—  at  least  in  hastily  written  and 
holistically  ai cored  timed  essay  tests. 
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CONCLUSION 


A  aajor  significant   finding  of  Part   II,   is  that 
some  students  do  in   fact   interpret  the  writing  task  in 
ways  which  differ   from  teacher-rater   task  interpreta- 
tions*    Yet  these  variant   interpretations  are  reason- 
able, considering  the  instructions  in   the  writing 
assignment.     The  greatest  mismatch  bet we  en  teacher  and 
student  expectations  and  evaluative  criteria  appears  to 
occur  when  students  are  assigned  the  proinpt  versions 
which  encourages  them  to  write  to  a  specified  audience 
other  than  a  general  or   teacher  audience.  For  example, 
when  students  write  letters  to  friends  or   parents,  or 
make  entries  in  personal  diaries,  many  of  them  value 
verisimilitude  in  such  instances,  and  they  attempt  to 
make  such  pieces  sound  more  like  **real  letters,**  than 
like  **real  English  themes.**  The  conflicting  frames  of 
reference  provided  by  the  simulated  life-like  communi- 
cative situations  and  the  actual  writing  context,  a 
classroom  test  occasion,   can  be  successfully  integrated 
only  by  the  most  able  students.     Even   then,   that  suc- 
cess may  be  questioned  by  student  raters  whose  under- 
standing the  nature  of  real  letters  and  diaries  does 
not  allow  them  to  accept  the  intrusion  of  any  hint  of 
school  writing,  which  is  seen  as  unnatural  or  showing- 
off.     Consequently,   there  are  disparities  between  the 
pupils*   conception  and  execution  of  assessment  tasks 
and   the  expectations  of  those  who  author  the  prompts 
and  rate  the  performance   in  response   to  them. 
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TABLE  1 

3':JRIN3   DI3TRI3  jri3M3  FDR    EACH  3R3UP  OF  RATERS 

^  F'JLL  5A,^PL£       VERSION   A       VERSION   a  RELIABILITY 

STUDENT       X                      5.66  5.65  5.67  d  =  .073 

RATERS        TJ                      5.0D  4.  00  5.  00  r   r  .63» 

M                     5.53  5.55  5.62  N  =  109 

S.  D.               1 . a9  1.64  1.33 

n                 114  57  57 

NOVICE^       X                     5.95  5  .  67  6  .  26  d  «    .  150 

TEACHER       -a                  ,    S.OO  6.  00  6  .  00  r   =  .55» 

RATERS        A                      6.01  5.73  6.19  N   s  106 

3.D.                1.33  1.40  1.20 

n                  105  55  51 

EXPERT-       X                      6.18  6.09  6.23  d   =  .044 

RATER3        1!                      6  .  00  6.  00  5  .  00  r   =  .71» 

M                      5.13  5.10  6.29  N  s    1 14 

S.  D.                 1  .  27  1  .  29  1  .  25 

n                 114  57  57 


d   =   •jiscrepan cy  rating:      proportion   of  total  essays 

re^ljlrtng  a  tnird  reading, 
r   =   Pearson's  r. 

H   i   total   papers  jsed   In  calculating  d  and  r. 
•   =   statistically   significant,  p£.01. 
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TABLE  2 

kZ^lZAln:  SEI^EEH    JKDJPo   3F  RATERS 


Rating  Group's  Scoring       Pearson's  R  s       Signifls^nt  H 
Sane  Samples  : 


otjJent  raters  witn  .^7  .031  ^05 

novice  teachsr  raters 

Stjdent  raters  with  .50  ,031  11 « 

expart  teacner  raters 

Novice   teacher  with  .6.*^  •031  105 

expert   teacner  raters 
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Table  2.1 

Dltcrlbuttono  of  Scorti  Awardotl  by  j  Grptipt  of  Kjftrt 


X  -  S.93 
S.O.  -  1.33 
It  -  IM 


X  -  S.66 
5.D.  *  i.*9 
R  -  lU 


2or 


TIT 


— r  V  ( — r 

ltikk«t  Uttri 


■^TT — I — 5 — I — 7 — r 

t«sch«r-llovlct  tLattri 


S.D.  -  1.27 
K  -  lU 


rrr 


IT 


Lit  &IX 


7 — r 


4      5  » 


TtXTAL  MOLf5T2C  !;r£lllLS  <lteri»<ta4  shov«:    2  -  ^rttt;  I  -  •trvnfcst  vritiDg. 
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Figure  1 

Facsimile  of  Grid  for  Period  Five:  32  Students 
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*  Not  all  students  voted  on  every  anchor. 
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APPENDIX  A 

COMPARISON  OF   SCORES  GIVEN   DURING  TRAINING 
BY  THREE  GROUPS  OF   RATERS  AND  THE  RESEARCHER-TRAINER 


Each  of  the  student  papers  used  as  an  anchor  for 
training  the  three  groups  of  raters  is  transcribed 
below  in  full,   exactly  as  it  appeared  to  the  raters. 
Following  each  essay  is  a  su.umary  chart  showing  distri 
bution  of  rankings  given  by  each  group  for  this  partic 
ular  essay.     These  rankings  are  followed  by  a  qualita- 
tive description  of  rater  responses  to  the  essay, 
including  the  researcher/trainer's  evaluation  of  the 
paper's  strengths  and  weaknesses  which  influenced  her 
choice  of  the  paper  to  represent  a  particular  score 
during  training. 
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APPENDIX  A 


SAMPLE  A 


As  I  look  back  on  my  attempts  it  new  and  different  things,  one 
experience  that  sticks  out  and  under  the  rest  Is  futHe  effort  at  driving 
a  car  with  a  manual  transmission.    "The  Clutch/  the  dreaded  part  of  the  car 
for  beginners,  causes  more  fits  of  flustered  frustration  than  any  other 
mechanical  device.    It  seems  to  sense  a  first-timer  and  tends  to  be  a  little 
bit  tighter  at  first  as  compared  to  when  you  get  familiar  with  it.    I  remember 
trying  to  stall  the  car  one  time  at  a  stop  sign  just  to  prove  to  me. that  It 
was  the  "Clutches"  fault  for  all  of  those  mid-Intersection  sta'lls  it  pulled 
on  me.    I  know  they  try  to  make  your  engine  die  when  your  trying  to  Impress 
the  girl  you  like  whose  waiting  at  the  crosswalk,  and  they  try  to  make  you 
lay  a  little  layer  of  rubber  on  the  road  when  a  cop  Is  In  close  vicinity. 
Everybody  knows  all  of  the  tricks  a  cluth  pulls. 

Unless  yoy  like  the  harmony  of  ten  different  car  horns,  don't  don*t  try 
a  main  intersection  on  your  first  day  on  the  "clutch."    Its  been  over  a  year 
now  that  my  mom  happened  to  bring  me  up  form  a  skate-boarder  to  a  "Driver." 
We'll  set  aside  the  countless  of  times  Tve  popped  the  clutch  or  stalled  the 
damn  thing.    None  were  more  memorable  than  the  stunt  I  pulled  at  the  Blossom 
Hill— Los  Gatos  Blvd.  Intersection.    I  was  the  first  in  line  at  the  left  turn 
Signal  with  the  simple  thought  of  making  a  U-turn  when  the  light  turned  green. 
I  was  easier  that  done  back  then,  and  as  the  light  turned  green,  my  sweaty 
hands  shoved  the  car  in  gear  and  the  jittery  feet  didn't  quite  put  the  "ole 
gas  and  clutch"  combo  together  right  and  the  car  the  preceded  to  advance 
nowhere.    There  was  a  non-melodious  choir  of  horns  to  follow  and  I  then 
attempt  to  go  aga^n,  only  to  do  an  encore,  then  finally,  as  the  light  turned 
yellow  sacrificed  the  clutch  and  burned  out  of  the  lane  and  left  20,  angry 
motorist- 
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SCORE: 

3 

3 

3 

3 

Discussion  of   Sample  A 

Researcher/Trainer  Response ,      This   paper  seemed 
typical   of  many   essays   in   the   sample  which  might   be  appro 
priately   rated   3  because   of   Its   particular   combination  of 
flaws   and   strengths.      This   Is   the  voice  of   a  confident 
writer  who   is   clearly   enjoying   the   assignment,  attempting 
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humor  with   some   success,    capturing   the   frustrations   of  a 
familiar   experience.      There   is   superficial  carelessness, 
but   there   is    also   ample   evidence   of   good   linguistic  con-- 
trol   so    that   dropped   letters   or   misspellings   might  fairly 
be   regarded    as    "slips   of   the  pen"  certainly   they  do 

not   interfere   with   communication.      Even  so,    the   number  cf 
purely  mechanical   faults   would,    in  most   assessments,  bar 
this   paper   from  a   top   score.      In  addition   to  cosmetic 
flaws,    organization   is   loose,    a  result   of   a  very  nice 
at  t  emp  t    to   balance   first-time   troubles   against   all  the 
ongoing    troubles   a   car  with   a   clutch   can  give.      The  "first 
time"   is   really,    in   this   writer's   mind,    a   series   of  exper- 
iences ,    summed   up   as    "trying   to   drive   a   car  with  manual 
transmission."     As    if  he   senses    this,   he   finally  focuses 
in   the    second   paragraph  on   a   single   incident:      "None  were 
more  memorable   that    the   stunt   I   pulled   at   the  Blossom  Hill 
...intersection..."   Actually,   he  may  have  had    this  episode 
in  mind    all   along,    but  he  has    set   it   among  several  gener- 
alities   about    the  war  with   the   clutch   that   all  inexperi- 
enced  drivers   must    face         a   perfectly   good  rhetorical 
device   for   solving   the  given  problem. 

S  tudent   Rater   Response .      The  students   were  spread 
over   three   scores   in   their   evaluation  of   this   paper  no 
one   thought   it   was    the  poorest  paper.      Those  who   gave  it 
2's   were   reacting   almost   exclusively   to    their   feeling  that 
the  writer   did   not    address    the   topic   properly   by  writing 
a   clear   account   of   a   first   time   experience.      One  commented, 
"I   couldn't    tell  what  he  was   really  writing   about."  hore 
favorably   impressed   students    argued   that   of   course  one 
could    tell  what  he   was  writing   about:      driving   a   car  with 
a   clutch.      And    the   majority   agreed   that  he  wrote   about  it 
well  --   they   liked    the  writer's   humor   and   details.      It  is 
clear   in  retrospect    that   for   all   groups   of   raters,  inclu- 
ding  the   res  ear cher / trainer ,    this   paper  might  have  received 
the  highest   score  were   it   not   for   the  number   of  mechanical 
flaws   and  were   it   read   in  comparison   to   any  other  papers 
except   Sample  J. 

Exper  t   Teacher  Rater   Response .      One   expert  argued 
well   for    the   strengths   of    this   papern   the   other    three  con- 
curred   that    the  mechanical   flaws   should   prevent   it  form 
receiving   the   highest  score. 

Novice   Teanh  er   Rater   Response .      Teacher  novices 
were   spread   over   all   four   scores,    as   might  be  expected 
given   the  variety   of   their   backgrounds   and   training  in 
English    together  with   their   lack  of   experience   in   the  class- 
room.     The  very   low  scorers  were   responding   almost  entirely 
to   the  mechanical   errors    in   this   paper,   which  provided  an 
opportunity   to   explain  the   role   of   error   in  holistic  assess- 
ment,   to   remind    them   that   errors   accompanying  hasty  writing 
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on  a  timed   test   but   that  do  not   interfere  with  communica- 
tion may   be   eval ua t ed  d i f f er ent ly   from  errors   on   a  care- 
fully edited   paper,    or  errors  which  make  comprehension 
difficult.      '.he   teacher  novices   who   liked    this  paper 
argued   for   it  well,    with   several   pointing   out   that  humor- 
ous writing   is   difficult   and   that   they  had   very  much 
enjoyed   the  voice   and  personality  of    the  writer. 
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SAMPLE  B 


The  first  time  I  ever  got  drunk  wis  back  In  eighth  grade.    The  reason 
that  It  Is  memorable  for  me  Is  because  It  was  such  i  weird  feeling.  Me 
and  a  couple  of  friends  went  up  to  a  party  that  this  girl  that  we  knew  was 
having.    She  had  a  couple  kegs  of  beer.    I  had  one,  and  thought  that  It  wasn't 
all  that  bad,  but  nothing  was  different.    I  had  a  couple  more,  and  I  started 
to  feel  a  little  bit  strange,  but  really  thought  nothing  about  it.    I  really 
did  not  think  that  I  got  drunk,  until  the  next  day  when  a  couple  people  I 
talked  to  were  telling  me  some  of  the  things  that  I  was  doing,  I  didn't  even 
remember.    This  has  not  become  a  habit,  I  get  drunk  maybe  every  two  weeks  or 
so.    Now  I  know  how  I'm  going  to  feel,  so  that  Is  why  I  can  retnembcr  the 
first  time  that  I  got  drunk. 
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Discu  ssion   o  f   Sample  B_ 

Researcher /Trainer   Response.      This   paper  was  my 
anchor   choice   for   the   lowest   score.      Among  papers   in  the 
sample,    it   was   one   of   the   least   compelling   on   a  quick 
reading.      The  vagueness   of   the   expression,    "it   is  memo- 
rable  because   it   was   such   a  weird   feeling"   is   never  re- 
lieved with    the   kind   of   clear   description   that   would  show 
us  exactly  what   the  writer  really  experienced.      As  it 
turns   out,    wha  t    is   most   memorable  about   the  experience 
is   that   the  writer   couldn't   remember   the  experience. 
Often,    the   writer  may   set  himself   a   difficult    task,  as 
in   this   case,    in  which   the   choice   of   episode  presented 
a  difficult    technical   problem:    what   was   interesting  to 
the  writer--the   fact    that   he  couldn't   remember  anything-- 
is  hard   to   write  about   interestingly   for   the  reader.  In 
spite   of    these   disclaimers,    the   paper   was   boring   to  read 
and   had   few   if  any  redeeming   features.      The   language  is 
fairly   flat   and   empty,    the  organization   loose   and  unde- 
veloped . 

Student   Rater   Response .      Student   scorers  easily 
identified    this  paper   as   one   of   the  poorest.  However, 
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its   defenders  were   vocal,    and    its   defense  consisted 
almost    entirely  of    such   comments   as,    "I   really  know^^how 
that    feels."      One    student  who   voted   for   a   score   of  "2" 
waxed   eloquent   about   what   happens    the   first    time  one 
get-   drunk,    and   all    the   things   you   learn  afterwards 
about   what   you   did.      This  oral   presentation   of    the  sub- 
ject  provided    a    fine    example   of   what    this   paper  needed 
to   have   been    in   order    to   receive   a   higher    score,    and  it 
helped    students   recognize    the   difference   between   a  paper 
they   liked   simply  because    it    is   about   a    familiar  expe- 
rience  and   a   paper    that   presents   that   experience  vividly 

Exper  t    Teacher   Rater   Response .    Surprisingly,  the 
experts    chose   only   one   paper    among    the   anchors    to  give 
a   score   of   "1"   and    it   was  not    this   one.      This   paper  they 
compared    to    Sample    C    in   quality,    rating   both   papers  "2". 
Having    encountered    100%   agreement,    and    following  ETS 
training   conventions,    I   did    not    insist   on   a    great  deal 
of   discussion,    allowing   Samples   B   and   C   to    stand  as 
representative   of    the   second    level    score.      I  was  then 
no    longer    so    convinced    that    they   did   represent  different 
rating    levels,    as   I   had   been  by  my   initial    impression  of 
the    sample.      However,    after    the   expert   reading,    as  I 
became    increasingly   aware   of    how   similar    in  quality 
were    the   papers   ranked    1    and    2,    how   few   1  scores^were 
being   awarded   by   raters    (because   of    that  similarity), 
and   how   large    the   range   of    quality   was   for   papers  awarde 
a   score   of   3,    I   realized    that   a   better   division   of  scale 
levels   might   have   ranked   Samples   B,    C,    and    I   at    level  1, 
forcing   readers   to  make   a   different   distribution   of    2  an 
3    1  eve  1    scores . 

Novice   Teacher   Rater    Response ■      This   group  was 
distributed   very  much    like    the    student   raters,  most 
agreeing    that    this   was   one   of    the   poorest   papers    in  the 
sample,    but   a   few   saying  merely   that   they   **liked"  it, 
that    it    described    a    common    experience   accurately.  The 
one   high  rater    recognized    the   potential    cleverness  in 
taking    a    subject  where    the   point    is    that   nothing  was 
remembered . 
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SAMPLE  C 

Tues . ,  dec.  3 

Dear  Diary, 

This  miy  sound  stupid,  but  I  give  a  speech  today  and  It  scared  me  half  out 
of  my  Witts.    I  have  given  speeches  before  but  this  time  I  could  only  use  notes 
instead  of  reading  it.    I  was  only  allowed  five  note  cards  and  It  had  to  be 
3  minutes  long!    I  went  last  and  had  the  whole  period  to  get  even  more 
nervous.    When  I  finally  got  up  there  my  knees  were  shaking  and  my  speech 
fnded  up  being  one  and  a  half  minutes  Instead  of  three.    Oh  well,  I'm  Just 
glad  It's  over  with.    My  teacher  says  that  we  have  to  do  another  speech 
next  semester  but  I  don't  want  to  think  about  it.    this  one  was  bad  enough. 
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Discussion   o  f   Samp le  C 

Researcher/Trainer  Response .      Initially  this 
paper  seemed   to   deserve   a  score   of   2,    for  being  slightly 
superior   to   Sample  B   only  because   the   tone   or  register 
and   the   sparseness   of   development   seemed  justified  by 
the   diary   form.      I  was   interested   in  wh ether   this  form 
would   appeal    to   student    raters   and  whether   it  would 
justify   to   experts   the   rater   unelaborated  recounting. 
The   piece   achieved  ve  r  i  s  imi  1  i  t  ude--- i  t    sounded   to  me 
exactly    like   what   a   student   might   enter   in   a   diary  the 
night   after   such   an   experience,    and   it   cleverly  sig- 
nalled  that   the   experience  was   the   first   of  raany--a 
constraint    the   prompt    suggests.      There   is   not   much  more 
here   in   the   way   of  vivid  writing   than   in   Sample   B;  the 
concrete    images   are  mostly   cliches*--'* scared   out    of  my 
wits,"   "knees    shaking."     I  was   prepared   to    leave   up  to 
the   rating   groups   the   decision   about   whether   to  reward 
this  paper  with   a  better   than   bottom   score   because  the 
language  was   appropriate   for   the   diary   form.  Whatever 
decision   was   made  would  become   binding   for   other  such 
papers . 

Student   and  Novice  Teacher  Rater  Response .  It 
was   precisely   the   dilemma   created  by   the   fact   of  the 
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diarv   form   that    caused   a  wide   division   among  studentsin 
evaluating   this   paper.      Its   defenders   insisted   that  it 
sounds  just    like   a   diary,"   and   they   went   on   to   add  that 
the   description   of   how  her   knees   shook  and   how  her 
speech   lasted  onlv   Ih  minutes   really   let   us   know  how 
the   aut'hor  was   feeling,    as   compared   to   the   writer  of 
Sample   B.      This    sample   was   chosen   as   a   representative  I 
by  all    student   scoring  groups. 

Novice   teacher   rater   responses  were  distributed 
in  much   the   same  way   as   the    students.      They   saw  this 
paper   as   superior   to   Sample   B   for   reasons    similar  to 
ones    given   by    student  raters. 

Expert   Teacher   Rater   Response .      The    experts  did 
n-t    make    a   distinction    in   quality   between    Samples   B  and 
c"      This   paper    seemed    a    clear   2    to    them,    but    so    did  the 
preceding   one.      Both   of    these    seemed    to   them  superior 
to   Sample   I,   which   they   awarded   a   1.      They   did   not  com- 
ment   in   discussion   that    the   appropriateness   of   the  _ 
language   for   a   diary   entry   could  be   viewed   as   a  special 
strength   of    the  piece. 
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SAMPLE  0 


So  I  hear  that  your  going  to  play  i  guitar  for  the  first  time.    I  know 
that  at  first  you'll  problbly  think  that  you're  never  going  to  be  able 
to  play  the  guitar  successfully  but  don't  let  this  discourage  you.    I  felt 
the  same  way  when  I  started  but  I  kept  on  practicing  and  now  It's  easy  for 
me.    At  first  you  may  find  that  changing  from  chord  to  chord  Is  the  most 
difficult   your  fingers  will  tie  up  In  knots  running  around  themselves.  The 
best  way  to  overcome  this  problem  Is  to  use  a  very  simple  two  or  three  finger 
chord  and  just  strum  the  strings  as  you  move  each  finger  In  unison  up  and 
down  the  six  strings  on  the  same  fret  and  then  do  the  same  thing  on  different 
frets  over  the  fret    oard;  put  your  fingers  in  an  A  position  on  the  second 
fret,  strum  a  couple  times,  move  your  three  fingers  up  to  the  fifth,  forth, 
and  third  strings,  strum  a  couple  more  times,  move  back  down  to  the  A 
position,  strum,  and  then  move   your  whole  hand  up  to  the  fifth  fret  and 
repeat.    Soon  you'll  find  playing  the  guitar  just  as  comfortable  as  It  is  fun. 
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Discussion   o f   Sample  D 

Researcher/Trainer  Response .      Samples   D,    E,  F, 
G,    K,    L,    M  were   set   aside   for   second-round  anchoring, 
as   a   check   on  where  the   cut-off   should   come  between 
scores   of   2   and    3.      None   of   these   seemed   to  be  either 
bottom   or   top   papers.      Sample   D  was   particularly  inter- 
esting  for   training  purposes,    since   it   was   clearly  a 
response   to  Version  B   of   the   prompt   which  specified 
audience   and   rhetorical   purpose  more   precisely.  This 
student   writer   responded  with   a   process   paper,    a   set  of 
directions   on   how  to   do   something   for   the   first  time, 
as   is   quite  appropriate   given    this  version   of   the  first- 
time   prompt.      The   problem,    of   course,    is   that  compared 
with   accounts   of   personal   experiences,    this   kind  of 
paper   seems   dull   to  many   readers--unless ,    of  course, 
one   is   interested   in   learning   to   play   the   guitar.  I 
was   curious   a^   to   how   students  would   evaluate  this 
paper;   would   they  verify  my   sense   that   these   are  good 
instructions--clear,    accurate,    helpful?     I  tentatively 
assigned   this   paper  a   3,    recognizing   that    it    does  not 
make   exciting  reading,    but   believing   that   it   does  well 
what    it    set   out    to  do. 
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Student   Rat  er   Response .      Students   as   a  whole 
were  not    favorablv   impressed   by   this   paper   compared  to 
others   in   the   sample.      The  majority    (52%)    scored   this  a 
2,    although   the    feeling   differed   in   the   two   classes  that 
were   given   a   second   day   of   training.      In   each  class 
students   responded   to  my   questions   about    the  accuracy 
and  helpfulness   of   these   instructions   qua  instructions 
with   fairly  noncommittal   comments   such   as,  "They're 
all   right.      They're   not   wrong."      Since   few  of  these 
students   attempted   to  write   precise   instructions  for 
this   test,    they   may   have   little   understanding   of  the 
difficulty   of   the   task;    certainly,    they  had   very  few 
examples   of   such  writing  with   which   to   compare  this 
piece   in   order   to   discover   its   relative  merit.  This 
situation   points   up   the   problem  of   dealing  with  dif- 
ferent  kinds   of   writing   in    a  holistic   assessment  and 
supports   advocates    of   primary   trait    scoring  who  argue 
that    different   kinds   of   writing  have   different  primary 
traits   and   cannot   be   fairly   compared   in   a   single  rank- 
ordering. 

Expert   Teacher  Rater   Response .      Only   one  of 
these   experienced   raters   considered   giving   this  paper 
a   2,    and   she   was   uncertain.      Given   the   anchors   from  the 
previous   training   session,    the   experts   felt   this  paper 
was   clearly   as    adequate   as   H   and   N    (the  mushroom  paper 
and   the    airport   paper)    and   better   than   B   and   C  (first 
drunk   and   first    speech   in   class),    to   which   they  had 
given   2's.      Agreement   was   strong   on   a   3   ranking,  so 
there  was   little   discussion   of   the   paper;    no   one  com- 
plained   that    it   was   difficult    to   judge   this  informative, 
"how-to"   paper   against    the  more   entertaining  personal 
nar rat  ives . 

Novice   Teacher  Rater   Response ■      This  group 
responded  more   positively   to   this   piece,  possibly 
recognizing   the   difficulty   of   writing   clear  instructions 
about    a   complex   process.      To   some   extent,   judging  from 
comments,    these    raters   rewarded    the   writer   for  coherence 
pleasant   audience   approach,    personable   voice,    and  com- 
plex but   controlled   syntax   during   the   lengthy  single- 
sentence   instruction  mid-paragraph.      There  was   a  strong 
consensus   that    this   paper   deserved   a   3,    but   no   one  con- 
sidered  it   a   4,   while   several   thought   it   deserved  less 
than  3. 
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SAMPLE  E 


Mom, 

We  ire  leaving  here  In  ibout  four  hours.    I  had  the  most  exciting 
vacation.    This  Is  the  first  time  I  have  ever  been  away  from  the  family 
alone.    Even  though  my  friend  came  with  me.  It's  still  different.  You 
add  so  much  security  when  we're  1n  foreign  places.    I  run  Into  a  distant 
stranger  every  second.    I  miss  you  and  Dad  because  I  never  have  to  worry 
aboL't  a  thing  when  you're  near.    I  have  learned  alot  this  trip  because  I've 
had  to  worry  about  money,  my  car  and  eating.    Don't  misunderstand  me, 
I  need  to  do  this.    I  feel  as  though  I  am  Just  beginning  to  grow  up.  It's 
like  a  young  »||»rrow  leaving  his  mothers  nest  to  go  out  Into  the  world  and 
defend  his  own  life.    It's  a  very  scarey  feeling  but  a  good  one,    I  have 
enjoyed  myself  very  much  and  It's  been  a  great  experience.    Thanks  for 
keeping  me  under  your  wing  till  I  was  ready  to  take  flight.    I  look  at  this 
as  one  step  towards  Independence.    Thanks  also  for  trusting  me  to  be  alone 
and  being  so  understanding. 


-your  daughter — 
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Discus  slon   o  f   Sample  E 

Researcher /Trainer   Resp  on  se .      In   spite   of  the 
apparent    cliche,    '*like   a   young   sparrow  leaving  his 
mother's   nest,"   and   the   general   lack   of   specificity  in 
the  writing,    I   tentatively   scored  this   paper   a   3  rather 
than   a   2   because   I   regarded   it    as   doing  effectively 
what   it    set   out    to   do.      Like  Sample   C    (the   diary  entry) 
this  writer   creates   an   absolutely  believable   letter — 
one  which  might    be  written   at    the  end   of   a  journey, 
after   other   letters  which  were  more   like  travelogues. 
This   one   would   be   intended   to   sum  up    the  meaning   of  this 
extended   "first    time"   experience.      After  hearing  the 
experts   discuss   this   paper,    I   became   more   aware   of  its 
weaknesses   and   see   that,    had  we    scored   this   paper   2  and 
lowered   papers   B   and   possibly   C   to   scores   of   1,    we  might 
have  made   a   broader   distribution   of   papers   and   been  more 
precise   in   our   use   of   2   and   3   as   scoring  categories. 
Nevertheless,    it    is   clear   that    this   paper  presents 
evaluation   problems    in   comparison  with   other  papers 
because,    like   Sample   A,    it   deals  with   a   so'-called  first 
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experience  which  was   actually   a  series   of  experiences, 
and   it   deals   with   that   expe-ience   on   an  abstract  level. 
There   is   definitely  more   attempted   here   than   m  the 
"Dear  Diary"   of   Sample   C  or   in   the   rather   flat,  repe- 
titious  recounting  of   a   first    day   at   school   in   Sample  G. 

Student   Rater   Response .      Students   were  divided 
fairlv   evenly   over   three   scoring  categories,   with  the 
bulk   (48%)    concentrat ing- on   a   score   of   J   for   this  paper, 
Dut   with   a   strong  minority    (23%)    civing   it   a  2   and  an 
equally   large   group    (25%)   giving   it    a  A.      Lower  scoring 
students   didn't   have  much  to   s  .-.y  about   this   paper:  i 
didn't    like   it.      1  do-n't   believe   she   did   that  (traveled 
alone).      There   aren't    enough   details--she   ought    to  tell 
something   about    a   time   when    she  wished   her   Parents  were 
there,    but   managed   to    get   herself    out    anyway.        The ^  _ 
latter   comment   is,    of   course,    a  strong  and  valid  criti 
cisn   of    this    paper.      Higher    scoring   students  defended 
this   paper    as   being   a   believable    letter,  ^"^^^^^^18^^?^ 
primary   criticism  of   lack  of   details   by   saying.      That  s 
not    the   kind    of    letter    it   is.      She's   writing   at^^the  end 
of   the   rrip--she   wrote   that    other   stuff   before.  This 
view  reflected   my   own   view   in    choosing   to   give  this 
paper   a   score   of   3.      Students   had   little   difficulty  in 
seeing   that    the   paper   was   not    strong   enough    for   a  top 
score,    reaching   consensus   at    3.      One    interesting  com- 
-^ent    in    favor   of   the   paper  was    a   compliment   paid   by  one 
student    in   response    to   the  metaphor    of   the  sparrow 
leaving    its   mother's   n e s t-- r emind in g  me    again   that  what 
are   cliches   to   adults   may  be    fresh   images   for  students. 

Expert   Teacher   Rater   Respo-nse.      This   group  sur- 
prised ^;^ri7ilh~3   out    of   4   readers   giving   this   paper   a  2 
after  having   used   the   3   category   so   freely.      I  asked 
them   to   defend    their    low  scoring   in    this    case        Line  by 
line    they   pointed   out    the   rather    empty  generalizations, 
some    awkwardnesses.      They  were   put    off  ^ ^ ^ ^ ^ 

They   were   not    interested   in    considering   the    letter  tor 
mat    or   the    special   voice   of    the  writer  which  might  be 
assumed    to   result    from   the    context    of  writing   a  letter 
to   a   parent.      Given    that    context,    they   felt    the  ^tuden 
could   or   should   have   written   more   effectively--and  that 
this   paper  was   parallel   in   quality   to   Samples   C   and  G 
rather'than   to   Sample   A,    for  example. 

Novice   Teacher   Rater   Response.      The    low  raters 
in   this'T^P   expressed   the   feeling   that   this  entire 
paper  was   a   cliche.      They   felt   superior   to   this   student  s 
struggling   effort    to    express    emerging   feelings    of  inde- 
pendence.     Fortunately,   a  high  rater  pointed   out   what  I 
lad   learned    from  the    student    responders:   what    seems  trite 
to   us   may   be   very    effective    and   new   to   students    at  this 
age.      These   readers   agreed,    however,    that    this   writer  is 
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not    giving  any   reader  new  or   deeper   insights    intu  v:hat 
it   means   to   gain    independence;    at   best,   we  have  a  snail 
portrait   of  how  one  adolescent    responded  with  gratitude 
to   being  given   responsibility.      Certainly  not    a  4  paper, 
but   the  group  was   happy   to   agree  on   3   instead  of  2. 
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SAMPLE  F 


T>,„  .tni  smell  the  smoke.    It  lingers  in  our  hair  and  clothing. 
The  L"Kli;rcl"-^""9  fl-f  a'nd  d.rV  ^/^^f  J ^rrrtlliafrin  ^fights 

Yet  that  came  later.    At  first  their  were  only  those  bright  tncKermg  g 
and  faint  roaring  noises,  hardly  threatening. 

Mo-n  wke  us  as  Dad  called  the  fire  dep't.   Their  roon,  was  closest 
to  the  garage     She  pounded  on  our  doors  shouting  fire,  telling  us  to 
get  outTf  thi  house    Pepper  reacted  first.    His  barking  and  Mo^'s  urgent 
tone  at  first  confused  me  then  her  words  sank  In.  Fire.  Firel 

Anne  »  I  scrambled  out  of  our  beds  and  grabbing  Pepper  hurried  out  of 
the  hSSle     The  smoke  made  everything  hazy,  fantastic  and  yet  real .  •  We 
stayed  at  a  friend's  and  talked  to  pass  the  time,  catching  up  on  the 
eight  months  in  which  we  hadn't  seen  each  other. 

Oad.  Mom  »  B  stayed  behind  to  save  the  furniture  '""^  P«7?;"^\''^;°?9- 
ings  thit  they  could.    The  f1re  consumed  the  garage,  part  oi'  Mom  &  Dad  s 
room  and  some  of  the  kitchen  i  bathroom.  When  hnK   
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.ELlJ-5-HASA2il  5^  Sample  F 

Researcher/Trainer   Res£onse.      In  all   training  for 
holistirTTTTHn^nt     I  usually   try   to   inclu  e 

::;:ar"unririS   ^"^'rranrto^Ile^t'th!   ,ron,  to  this 
type   and  arriie   a?   a  policy  decision   about   how   to  score 
type   ana  a  alternatives   are:    (a)    to   imagine  what 

such   appers.      The   alternatives  k    j  simply 

^hf   ap  r     n  thraiaila^h^fevidence^f 
t"tii  "Mlit""h^t\ot%Lalizing   it   ^°       t  hr^atfercr  ^  ^ 

llles       By  'either  policy,   I  found  the   P^-^  I, 
^   at  -least        The   absence   of  closure  prevented   a   score  ot 
I  under     okcy   (b),   but   the  strength   of  the  writing  made 
it    interesting  enough  reading   to   earn  a 
Jould   not   award   it   a  4   under   policy     a),   ^^fing  its 
potential,   because   I  felt   it  was  unclear  "Aether  the 
student   c^uld  end   it   well.      The   last   paragraph  to  be 
lAtten  ^^ell  have  petered   out  because  the  student 
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couldn*t    figure   out   v;here   he  was   going   or   hcv;  to  luii- 
clude  what   he  had   begun   so  dramatically. 

Student    Rat  er   Response .      This   paper   earned  all 
four   scores,    the   very   lowest    scorers   basing   their  deci- 
sion  simply  on   the   fact    that    the   piece  was  unfinished, 
and   that    therefore   the   student   hadn^t    really   "done  the 
assignment."      It   was   easy   to   persuade   these   students  to 
abandon   this   all-or-nothing    formula   for   success   and  ask 
them   to   try   to   judge   the  writing   that   was   there.      A  few 
students    feXt   this   paper   included   some   of   the  best 
writing   of   all   the   anchors    (especially   the   first  para- 
graph),   and   so   they  wanted   to   award   it   a   4.      They  agreed 
to   a   lower   score   when   reminded   that   after   all'  the  writer 
had   not    finished    and   we   don't    really  know   if   he  could 
have    finished   well.      Finally,    the   debate   centered  on 
whether   the   paper   deserved   a   2   or   a   3.      Because   the  aim 
of   the   training   session   is   to   achieve   agree nent,    not  to 
establish   absolutely   the  merits   of   each   paper,  discus- 
sion  was   not   prolonged.      I   presented  my   view  that  there 
was   sufficient   writing   of   good   quality,    that   the  paper 
was   rewarding   to   read   even   without   an   ending,    and  that 
it   was   stronger   and   more   interesting   than   the  other 
papers  we   had   scored   as   2*s.      Students    seemed  comfort- 
able with   this    resolution   and  were   generally  willing  to 
take   the   view   that    unfinished   papers    can    be   judged  for 
what   they   offer,    without    trying   to   speculate   about  how 
well    they   might    have   been    completed   had    the   writer  been 
givenmoretime. 

Expert   Teacher   Rater   Response .      The  experts 
agreed   in    general  with   the   novice   teacher   rater   and  my 
response,    discussing   the   extent    to   which   the   absence  of 
an   ending   interfered  with   the   reader^s   ability   to  enjoy 
this    piece   and   understand    the   significance   of  that 
"first    time"   experience.      One   expert   felt    the   lack  of 
ending   to   be   greatly   detrimental,    resulting   from   a  -lack 
of   overall   planning   and   a   sense   of   purpose   in  waiting 
the   essay.      All    readers   felt    this   student   had  excellent 
narrative   skills   and   a   real    sense   of   how   to  begin  a 
dramatic  account. 

Novice   Teacher   Rater   Response .      There  was   a  wide 
range   of    responses  ,    in   spite   of   previous  training, 
because   of   the   missing   ending.      This   group,  however, 
produced   a   larger  percentage   of   top   scores   than  the 
student    raters   had   because   they  were  more   impressed  by 
the   rather   effect ive   writ  in g   of   the   opening  paragraphs. 
This    group   produced   good   criticisms   of   the   piece,  how- 
ever,   saying   that    the    sudden    loss    of    focus    and   the  tele- 
scoping  of   events    at    the   end   of   the   third  paragraph 
suggest    the  writer   had   abandoned   the  moment  worth 
writing   about    and   was    going   to   have   difficulty  finding 
good    closure — even   with    adequate    time   to  write. 
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I  remember  what  my  first  day  of  high  school  was  like  Pe;;^"^^^-.  ^ 
remember  how  frightened  I  was  that  I  would  so  something  stupid.  I  felt 
like  I  was  the  only  person  In  the  school  f  '"^^^'J  Worker 

else  knew  exactly  wh»t  they  were  doing.    Instead  of  going  to  my  locker 
between  each  class,  I  brought  all  of  my  books  with  me.    I  guess  i  was 
afraid  thJt  I  wouldn't  hav2  enough  time  to  go  all  the  way  to  my   ocker  and 
then  try  and  find  my  next  period  class.    Before  "ch  class  [^f^ 
practically  pray  that  there  would  be  someone  Inside  that  I  knew,  i 
remember  thinking  that  I  would  never  make  any  new  friends.    I  f e  t  so 
nervous  the  whole.    I'm  so  happy  that  day  1s  gone  .^'J?;  2^ 

forget  how  nervous  I  was  and  what  ■  relief  1t  was  when  I  heard  the  6th 
period  bell  ring  to  let  us  go  home. 
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Discus  sion   o  f  Sample 

Researcher/Trainer  Response.      This   sample   is  much 
like   Samples  B   and   C  in   level   of   detail   and  scope~-per- 
haps   a   trifle   stronger   than   these,   but   certa'inly  not  up 
to   Sample  H   (mushrooms)    in   overall   impact.      The  piece 
is   unredeemed  by   the  kind  of   lively   pointed   ending  that 
saves   Sample  N    (airport).      It   seemed   a   good   paper   to  use 
to   stabilize   scorers'    concepts   of   a   2   paper,    having  a 
little  more   substance   and   coherence   than   the  papers 
representing   the   1   score    (Samples   I   and   possibly  B) . 

Student   Rater  Response.      Students  were  split 
between   the   scores   2   and   3.      The   appeal   of   this  paper 
was   strong  and   apparently  based   in  part   on   its  subject: 
defenders   spoke  of  how  vividly   and  convincingly  the 
writer  had   captured   an   experience   they  recognized. 
"It's   lust   like   that,"   was  a   common   comment.  These 
defenders   rightly   saw  more   details   than   in  weaker  papers 
and  were  not   easily  convinced   that   the   paper  was  not 
strong  enough   for   a   3   until   they  were   asked   to  re-read 
Samples   A  and   H.      Even   then,    some  maintained   that  this 
paper  was   just    as   good,    and   I   allowed   the   difference  an 
opinion   to  stand,   pointing  out   that   every  assessment 
yields   some  borderline  papers,   and  that   the   only  scorers 
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who  were   really   off-base   were   the    l*s   and   4*s,  (This 
kind   of   training   is   dangerous,    of   course,    since  it 
encourages   scorers   to   stick  to  middle   scores   and  not 
trust   themselves   on   the  extremes.) 

Expert   Teacher   Ra t  er  Re  s  ponse .      These   four  raters 
were   unanimous    in   their   recognition   of   a   lower  half 
paper  with   some   redeeming   strengths   and   agreed   with  the 
trainer^s   assess me nt.      There  was   no   discussion   of  this 
paper. 

Novice   Teacher   Rat  er  Response .      By   the  second 
training   session,    this   group  was   better   at  spotting 
lower  half   papers   and   remaining   clear   in   the  reasons 
for   their  choice. 
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SAMPLE  H 


Mushrooms 

Two  years  ago  my  Spanish  teacher  took  us  out  to  lunch.    We  went  to 
Thunder  Mug.    It's  a  favorlle  Pizza  Parlor  my  old  junior  high  school  used 
to  go  to.    Before  we  left  we  had  to  decide  what  kind  of  Pizza  we  wanted 
to  get,  so  my  teacher  could  call  In  and  order  It  to  be  ready  when  we 
got  there.    There  were  five  people  In  my  group.    We  voted  on  what  kind  of 
pizza  we  would  order.    All  four  of  them  except  for  me,  wanted  a  mushroom 
pizza.    I  didn't  want  to  argue  about  It  so,  I  went  along  with  It.  But 
I  wouldn't  eat  the  mushrooms.    I'd  take  them  off.    When  we  got  there  the 
Pizza  was  ready.    I  saw  all  throughs  goobs  of  mushrooms  on  It,  and  thought 
I  would  get  sick.    We  found  a  table  and  they  started  eating.    I  took  a 
piece.    I  looked  down  at  It  and  all  thoughs  mushrooms.    I  though  of  the 
last  t1ne  I  had  had  mushrooms,    (those  sick  gray  things  that  tasted  like 
rubber  (all  mushy)  I  started  to  pick  them  off.    Every  body  sat  there 
staring  at  me,  and  I  felt  like  a  fool,  '.o  I  put  the  mushrooms  back  on, 
closed  my  eyes  and  took  a  bite  (try  not  to  taste  anything.),    I  tasted  good. 
I  couldn't  believe  It.    I  took  another  bite,  and  It  tasted  even  better. 
Soon  I  had  downed  more  then  my  share  of  the  pizza,  so  I  stopped  eating. 
From  that  day  on,  all  my  Pizza's  have  tons  of  mushrooms  on  them.    I  now  put 
mushrooms  (In)  on  Just  about  everything.    I  eat  them  on  Pizzas,  on  steak, 
sauteed.    (next  to  Just  about  any  meat.  In  salad,  mixed  In  vegetables.  I 
don't  see  how  I  couldn't  have  known  how  good  they  are.    I'm  glad  I  tried 
to  cat  them  two  year  ago,  or  I  still  would  not  know  how  good  they  art. 
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Discussion   of   Sample  H 

Researcher /  Trainer  Response .      In   the  initial 
selection   of   anchors,    I   considered   this   piece  inferior 
to   Sample   A,   but   superior  to   Samples   B,    C  and  I--and 
about   equivalent   to  D  and  L.      I  anticipated   eating  B, 
C,    I,   as   1  papers,   and   rating   Sample  H   (with  D  and  L) 
as   2   papers,    saving  the   score   of   3   for  papers  as  good 
as   Sample  A.      I   found   the  straight   narrative   style  of 
this  paper   rather   flat;   I  was  bored  at   the  beginnltig 
and   reluctant    to  keep   on   reading.      Only   during  the 
training   of   rating  groups  did   I  begin   to  value  this 
paper  for  the   clear  impact   of   its   simple   anecdote,  and 
I   found  myself   persuaded  to   raise   the   score   of  this 
paper  to   3.     Unfortunately,   when   that   shift   was  accom- 
plished,   a  good  many   other  papers  which  were  weaker  than 


^00. 


Sample   A,    our   first   well-established  3  paper,   were  also 
scored   3.      This   paper,    then,    and   its  popularity,  con- 
tributed  to   expanding   the   3   category  at   the   expense  of 
other   scoring  categories. 

Student   Rater   Response ,      There   was   no  arguing 
with   the   appeal   of   this   p ap e r a Imo s t    as  many  students 
wanted   to   give   it    a   top   score   as   wanted   to    give    it    a  2, 
with   the   bulk  of    scorers   preferring   a   score   of   3.  They 
couldn't    argu"e   about    the   comparative   syntactic  resources 
of   Samples   A  and   H--all   they   knew  was   that    they  had 
enjoyed   these   papers   about   equally,    and   that   both  were 
"more   fun'*   than   most   of   the   other   papers   in   the  sample. 
I    acknolwedged   tehir   certainty;    their   taste,    after  all, 
was   an   important   measure   of   the   success   of    this  piece 
of   writing.      The    few   students   who   gave   this   paper  the 
poorest    score   admitted   that    they  hadn't   finished  reading 
it-'-apparent  ly   the   opening  had   affected   them  as    it  had 
me  . 

Expert  Tea  cher  Rater  Response .  These  four  reader 
initially  were  quite  divided,  eventually  choosing  a  3  by 
consensus,    as   did   the   other  groups. 

Novice   Tea  cher   Rater   Response .      This   group,  like 
the   student    raters,    simply   enjoyed   this   account,  reading 
into   it    a  nice   lesson   on   how  we   learn   to   like  something 
when   we   are    forced   to   try   it.      One   pointed   out   that  the 
straightforward   narrative   style   was   a   successful   way  of 
organizing   a   response   to   the   prompt;    the   reader  always 
knew  what   was   going   on   and   didn't   have   to  make   any  leaps 
The   cumulative   effect   of   the   piece  was   strong,  whatever 
one   thought    about   particular   sentences   or   phrases.  The 
central   part   of   the   essay,   with   its    freely    and  rather 
carelessly   used   parentheses,    struck  manyreaders  as 
delightful;    they   liked   this   student's  "voice." 
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When  I  WIS  about  five  yeirs  old  I  put  some  Insect  poison  In  my 
mouth  not  knowing  that  It  was  poison.    I  later  found  out  that  It  was 
poison,  and  became  very  scared,    I  rember  hiding  by  myself  wondering  ^r 
I  was  going  to  die.    After  a  while  I  was  still  feeling  allrlght  so  I 
came  Inside,  went  about  my  normal  routine,  and  soom  after  forgot  the 
Incident.    It  embarassed  me  so  much  that  I  never  told  anyone  about  it, 
and  I  think  this  Is  why  I  still  remenber  It  so  well 
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Discussion   of   S amp  1  e  I. 

Re  searcher /Trainer  Respon  se .      This   paper  had  an 
appeal--a   kind   of   simple   dignity.      This  writer,  like 
the   one   in   Sample  B,    is   facing   the   problem  of  writing 
interestingly   about   not   experiencing   symptoms  he  could 
report    in  vivid   detail.      He  had   to   report   what   had  hap- 
pened  in   his   imaginat ion--how   it   felt   to   think  he  was 
going  to   die.      Hemingway  very   neatly   avoids   this  problem 
in   "A  Day's  Wait"   by   telling   the   story  through   a  father's 
eyes;    the   child's  behavior   is   described  as    simply   as  in 
this  writer's  account. 

Nonetheless,    this  writer   clearly   does  not  attempt 
much  and   is   satisifed   with  a  bare   recounting  of   the  inci- 
dent.     As   far   as   it   goes,    it   would  be  a  perfectly  accept- 
able oral   response  to   an   oral   request,   but    it   does  not 
recognize  the   contract    implied   in   a   written  test,  a 
contract   that   might   be   described  as   "show   off  your 
writing"    (see   Sample  J),    or   less   per j orat i vely  as, 
"write   so   that   we  can   relive   the   experience  with  you-- 
write  as   if  you  are   a  writer ,   not   a   test   taker   or  a 
respondent   to   a   simple   question   in   an   oral  dyad." 
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Student   and   Novice   Teacher   Rater  Re  s  p  on  s  e  « 
Student    rankings  were   somewhat   more    spread   on  this 
paper   than   on   Sample   B.      Again,    the   recognition   of  an 
experience  which   has   been   told   simply  and   clearly  and 
which   itself   carries   an   impact   appealed   to  many  students. 
The   preceding   discussion   of   Sample   B,   which  called  for 
more   details,    inspired   many    students   to   defend  their 
low  scores   for   this   paper    (as   did   the   experts)  by 
pointing   out    simply   the   lack   of  elaboration,    the  need 
for   "showing,    not    telling."     Novice   teacher   raters  had 
similar  reactions. 

Expert   Teacher   Ra t  er   Response .      The   experts  were 
unanimous   in   citing   this   paper   as   the   anchor   for  the 
lowest    possible   score.      As   with   Samples   B   and   C,  the 
unanimity   invited   little  discussion.      This   paper  was  a 
recognizable   type   to   the   readers   from  their  experiences 
in   reading   for   ETS:    the  writer   simply  doesn't  attempt 
enough. 
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SAMPLE  J 


Dear  Rachel . 

I  am  so  excited  that  vou  are  coming  out  to  visit  ne  this  surrer. 
The  first "hing  we  are  going  to  do  Is  go  to  Harriot's  Great  Ar.erica  and 
Hde  the  Tidal  Wave.    Remembering  that  you've  never  b.en  on  a  ^o'le-- 
coalter  before.  I  promise  I'll  be  patient  with  all  your  screams  and  pleas 
for  deferral.    But  don't  worry  because  I  retcted  the  same  way  the  first 
time  I  soared  through  the  skies  upside  down.    In  fact,  my  first  experience 
on  that  death-defying  machine  will  always  swirl  through  my  mind  as  tne 
rSner  coaster  whooshed  through  space.    Accompanied  by  three  other  friends, 
I  had  bravely  approached  the  long  line  extending  from  the  mock"  ship. 
Somehow,  waiting  the  forty-five  minutes  it  took  to  reach  the  front  of  the 
queue  was  far  easier  than  the  last  60  seconds  before  ny  turn  >^rl^f^;, „, 
Up  close,  the  noise  vibrated  the  wooden  platfom  more  so  thsn  the  actual 
rush  of  acceleration.    My  face  changed  .  different  hue  every  breath  1 
took,  which  ocurred  rather  frequently,  «s  1  stepped  ^ "91^,1"*° 

metal  death  chamber.    Hastily  the  amusement  park  employees  -""^t^d  their 
instructions  unheard  by  me  «s  I  recited  my  prayers     Before  my  '""se  God 
J  d  been  completed  .  giant  palm  swatted  the  back  of  the   ^«  "  '"J^  ^ 
it  forward  along  the  ominously  constructed  tracks.     My  head  yanked  back- 
wards against  the  head  rest;  my  eyes  clamped  shut;  .nd  ny  sweaty  palms 
acquired  a  grip  Impossible  under  normal  circumstances.    With  n  a  matter  or 
seconds  I  could  feel  an  unaccountable  force  piimping  all  ny  blood 
my  head.    Opening  my  eyes  for  an  explanation    I  discovered  a  t'-ans/°rnati°n 
1^  the  earth's  terrain-I  was  upside  down.    Pushed  through  a  muggy  S"^ner 
1  y,  the  sweat  dripping  down  my  face  flew  off  at  the  "rners     And    he  next 
thi^g  I  knew,  the  mechanical  caterpillar  jerked  to  a  halt,  and  I  was 
hu5?led  dizz;  and  dazed  from  the  train.    Smiling,  I  walked  away  saying. 
"Soy  thdt  was  fun  let's  do  It  again." 

You'll  love  1t  Rachel.    Can't  wait  to  see  you. 

Love 
J  
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Discussion   of   Sample  J 

Researcher/Trainer   Response.      I   chose   this  paper 

without   question   the    top    score   possible.      I  vas_onl> 
I  oncerned   whether   there  would   be   -/^^^h   papers     n  the 

f  sample   that   so   successfully   responded   to   ^he  task^ 

this   paper  were  too    far   superior   to   most   others  it 
would   create   too  narrow  a   scoring  category   for  the 
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re  of  A,  elir.inating  rrany  other  papers  which  should 
be  considered  top  papers  if  the  reading  were  to  ol:tain 
a   good   spread   of   papers   across   the   scoring  range. 

J  has   addressed   her   narrative   to   "someone   v:ho  is 
about    to  have   a   similar  experience/'   providing  a  realis- 
tic   rhetorical   context,    choosing   an   appropriate    fcrm  (a 
letter),    and  working  within   these   constraints   to  write 
a   particularly   vivid    account    of   her   own   first  experience 
on   a   roller   coaster.      The    flatness   of   the    first    few  sen- 
tences,   and   the   overwriting   of   sentence   6,    I  attributed 
to   problems    in   creating  the    setting-'-problems   pro mp ted 
by   the   task.      As    creator  of    the   task,    I   was   quite  will- 
ing  to   read   past    the   writing   problems    that    seemed  to 
result    from   the   constraints    I   put    on    the   writer  (osten- 
sibly  to  help  her  write  better!),    privately  congratu- 
lating  her   on  making   such   a    good   attempt    to   couch  her 
anecdote   in   the    form   invited  by   the   proirpt.      I  skimmed 
these    first    sentences   to   get    to   the   "real"   purpose  of 
the    assignment,    her   recounting  of   her   own  first-time 
experience.      I   was    soon   impressed  by,    "Somehow,  waiting 
the    forty-five   minutes   it    took  me   to   reach   the   front  of 
the   queue   was    far   easier   than   the   last    60  seconds..." 
and,    "the   employees    recited   their   instructions  unheard 
bv  r.e   as   I    recited   my   prayers...."      I   read   on,  looking 
for   a   good   solid   physical    sensation,    and   I    found  it, 
as   the   "giant    palm   swatted    the  back   of   the  train...." 
This   student   had    fulfilled   my   expectations,    and  con- 
tinued  beautifully   to   do   so    for   the   rest   of   the  essay 
c  um  letter. 

Student    Rat  er   Response .      I   was   startled   to  find 
that    students   did  not    universally   recognize   this  paper 
as   the   apex   of   achievement ,    and   they   had   good  reasons 
for   their   reservations.      The  most    striking  comment, 
offered   as   a   defense    of   the    low  mark   given   by   a  respon- 
sible   student    in   a   college   prep   literature   class,  was 
simply:    "It    sounds   too  much   like   an   English  theme." 
Other    students   spelled   out    this    criticism,    citing  line 
and   phrase    (especially   in   the  beginning   of   the  letter), 
complaining   that   no    one  would   ever  write   a   letter  like 
that.      I  believe   that    they   were   reacting  negatively  to 
two   different    problems   in    the  writing   of   J   on  this 
occasion.      The   primary   problem   is   the   one   created  by 
the   prompt,    "write    for   someone  who   is   about    to   have  a 
similar   experience."     The   students   were  more   harsh  in 
their  judgment.      Essentially,    they   said   that    if  this 
writer  wanted   them   to   believe   this   was   a   letter   to  a 
friend,    she    couldn't   write    it    like   a   composition   to  an 
English   teacher.      Fortunately,    for  purposes   of  training 
raters    to   reach   agreement,    there   was   at    least   one  student 
in   each   class   who   argued  that    it    could  be   a   letter   to  a 
friend.      The   classroom  teacher   intervened   to  inquire. 
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friends?"   and   received   a   g^^^"^'^/^^   students   to  recog- 
In   other  words,   we   "\P^°^^^^/'"g\o  value   and  give 
nize  that   English   ^/ ^  ^  ^  .  ^      a  che  r^  apj  r  ove  d  writing 
high   scores   to   good  ^sh-   eacher   aPP  ^^^^^^ 

even   in   the   context  is  unbelievable 

even   if   the   students   think   the  writing 


even 

in   that  context. 


making   is   one   they  had  no  wor  ^.^^^ 

HOW  the   novice   ^^^/^J  ^./.J^rg  /ach  in  e  "  wi  1 1' a  Iway  s  swirl 

rience   on   that   death   derying  whooshed  through 

than   for  effect. 

^^  M^-^  rpr^s^ntinr.- 
.aters  -ani.ously  chose  hi  PaP-^//,^^  ,  ^nd  did  in 
top   score,   wasted   littxe   ti  j^^^/trainer  was  con- 

fact    fall   into   the   trap   ^^^^"'/"^pe's   in   the  sample, 
earned   about:    ^i^^^^S   few  other  papers 

tnat  so  clearly  and  ;^^^^^i^3/rrained  raters'  criteria 
maker's  ^  -  ^ .^f,  ^  ^  °,  /  d  i  on  and  syntax,  the  flaw- 
for   success.      The   level   oi  celling  details,  the 

less  mechanics,  the  f J  '  ., ^  ^  r  in  g  devices  (moving 
play   en   "  -  c  i  t  e  d  .  .  •  r  e  c  1 1  e  d         h  e^c    ^^^J^^  ^^^^^^^ 

fron.  "mv  pravers  to     B  anecdote    in  a 

completed...    ).  ^   ^o   the   audience  constraint- 

letter   format   as   a   =  ^  J     J  °^  ,   °  ^  ^  3  ^   the   introduction  and 
and   then  --^^-\lYl,[Ta   r  e  a     1  e  1 1  e  r- al  1   of  these 
rriltiirinTne^piecrof  writing  make   this   an  excep- 
tional  paper. 

teachers   recognized  that   this  was   a"     PP^   ^^^^^^  ^^^^ 
but  they   split  very  "^'"^     „ith  30%  complaining 

Whether  it  was  ^^^^  °   /^,nt;r"io\  friend"   or ,  th  at 

fhat    it   wasn   t    a      oei^j-cva  ^       iust    don  t 

i^was   overwritten. One  ^/^^^     Ined     "She  '  s 

like   her  very  ^ili^y      rather   than  really 

showing  off  ^-^^^^^i^Ji^i'e  "     An   interesting  commen- 
writing  about  ^^^f  J',^  J'p^rf  orman  ce  evokes. 

^orv   on  what   a  writing   test  pen 


ER?C 

  306 


SAMPLE  K 


It's  funny  how  some  people  think  rwre  ibout  the  things  they  dislike 
thin  those  that  pleise  them;  one^d  think  every  body  would  prefer  dwelling 
on  their  virtues  ind  idvintiges  thin  their  short  comings.    I  dislike  writing 
under  given  topics.    I  live  for  the  present.    Thusly  from  out  of  i  digression 
I  confonn  to  the  subject  ind  describe  the  first  time  I  wrote  for  in  experiment. 

The  day  hid  been  mundane,  the  weather  warm,  my  lunch  to  large  and  none 
too  good;  upon  entering  my  English  class  a  stapled  set  of  papers  wa$  thrust 
on  me  and  I  was  sent  into  an  adjacent  room  to  do  whatever  It  was  the  sheets 
dictated. 

Now,  to  make  on  thing  clear:    Hy  ego  Is  at  odds  with  my  common  sense. 
The  latter  would  dictate  my  avoiding  the  wrath  of  Miss  McNelly  by  buckling 
down  and  doing  my  best;  the  former  compells  me  to  sneer  and  retreat... 
unjustifiably.    This  paper  was  begun  only  after  twenty  minutes  of  battle 
had  passed  between  the  two  factions  of  my  consciousness,  for  In  order  to 
write  up  to  my  standards,  I  would  have  to  say  something  Important  to  me... 
which  I  would  not  reveal  simply  for  the  sake  of  an  experiment.    A  fault 
of  mine,  I  suppose...!  am  too  reticent  towards  baring  my  innermost  soul 
to  those  people  who  simply  have  every    business  knowing;  my  peers  and  my 
te^ichers.    So»  I  take  a  sneaky-panther     cop-out—-  and  here  you  have  your 
data. 


STUDENTS 

TEACHER  NOVICES 

EXPERTS 

RESEARCHER 

n  i 

n  % 

n  i 

(one  reader) 

1 

2  (03) 

1  (07) 

0  0 

0 

2  18     (29)  7       (30)  0        0  0 


3  21      (34)  6       (26)  3     (75)  X 


4 

21  (34) 

9  (39) 

1  (25) 

0 

CONSENSUS 

3/4 

3 

3 

3 

SCORE : 

Discuss  ion   o f   Sample  K 

Researcher/Trainer   Response .      Perhaps   one   of  the 
most   intriguing  papers   in   the   sample,    this   one  was 
included   as   an   anchor   because   I  wanted   student  raters 
to   see  how  a   writer  might   argue  with   the   test  situation, 
protesting  his   "first-time"   role  as   a   guinea   pig  while 
still   technically  "conforming   to   the   subject"   and  writing 
an   acceptable   composition.      The  writer's   agonies,  however, 
do   take   their   toll   rhetorically:   I  would   rate   the  paper  a 
3. 

Student   Rat  er   Response .      Most   students  recognized 
the   strength   of   this   paper--a   good   number,   however,  gave 
it    low   scores   because   they   felt   it    didn't   make   sense  or 
wasn't   an   adequate   response   to   the   assignment.  Fortu- 
nately,   in   each  class   the   criticisms  were  heard  first, 
Inspiring  an  articulate  and   spirited  defense  by  some 
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studen^t   who   sympathized   with   the  writer's   dilemma  and 
was   enormously   impressed   with  his   method   of    solving  it. 
These    defenders   were   often   unwilling   to   accept   my  ver- 
dict   that   the   paper   had   flaws   that   would  make   it   a  3, 
and   so    I   let    it   stand   for   this   group   as   a  borderline, 
3/4  paper. 

Expe  r  t   Teacher  Rat  er   Response  .      All   four  e.^perts 
appreciated    fully  what    this    student   had   achieved,  but 
three    agreed   with   my   first   assessment   that   the  rough- 
ness,   especially   of   the   opening   paragraph,    and  the 
connections    the    reader  had   to   provide   throughout,  made 
this    less   than   a   top  score. 

Novice   Teacher  Rater   Response .      The  novice 
teachers  were   as    enthusiastic   as   the  positively 
impressed   student    readers,   with   critical  comments 
coming   primarily   from   those   who   seemed   not   to  have 
read    the   paper   carefully   enough   to   recognize   what  had 
been    attempted   and  achieved. 
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SAMPLE  L 


In  my  youth  I  hirdly  ever  got  s1ck,  but  when  I  did,  I  grew  very 
s1ck.    Such  was  the  case  when  I  first  experienced  a  hospital.  My 
temperature  had  climbed  far  iibove  normal  and  wouldn't  regress.  I  remember 
most  vividly  the  trip  through  the  halls  where  people  from  all  walks  of 
life  could  be  seen    Illness*  the  only  thing  that  they  had  In  common, 
brought  them  together,  each  of  them  unconcerned  with  any  otfier.    I  had  a 
hard  time  not  feeling  sorry  for  them  until  I  was  the  one  manipulated  by 
the  nurses.    Everyone  rushed,  while  I  hardly  could  move.    Hospitals,  which 
cure  one's  body^  also  tike  a  bit  of  one's  sanity. 


STUDENTS  TEACHER  NOVI CES  EXPERTS  RESEARCHER 


n 

% 

n 

i 

n 
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(one  reader) 
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0 
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2 
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6 
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1 

(04) 

0 

0 

CONSENSUS  3  2  3  2 

SCORE: 


Discus  s  ion   o f   S  ample  L 

Researcher/Trainer  Response .      I   chose   this  paper 
as  an   example  of   a   good   2   score,    remaining  at  too 
abstract   a   level,    lacking   development,    but   with  nice 
syntax   and   a   last   line  which   can   be   read   as  cohering 
all   that    goes  before. 

Student    Rater   Response.      Students  were  more 
impressed   than   I   was   by   this   paper,    and   judged   it    a  3 
in  the   one   class  which   had   time   to   read   and   rate   it  as 
a   group.      They   argued   strongly  with  me,    and   I  allowed 
them  this   one   as   being   a   little  more   imaginative  and 
effective   than   such  papers   as   Samples   B,    C,    or  G.  I 
wanted   to   encourage   them  not   to   consider   length  as  a 
s  ine   qua  non   for   high   quality,    and   it    seemed   useful  in 
training   to   allow  one   short   paper   a  score  of   3  based 
on   other  merits . 

Expert   Teacher   Rater   Response .      These   four  raters 
were   divided   evenly  on   this   paper;    two   of   them  reflected 
the   response   of  myself   and   the  novice   teachers,  two 
others    responded   as   the   students   had.      Defenders  read 
the   paper   aloud   pointing   out    that    it   was   tight,  consis- 
tent,   and   said   everything  that   needed   to  be   said  to 
support   that   last   line,    which   they   found   quite  effective. 

0  . )  . 
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Detractors,    my s e 1 P  in c 1 ud e d ,    complained   that    the  long 
sentence   quoted   by   the  novice   teachers   was   difficult  to 
understand,    rendering  unclear   the   point    of  view  of  the 
whole   piece.      Defenders   in   turn   accused   low   scorers  of 
careless   reiding.      The   low  scorers   replied  with  rnaxims 
bout    the   responsibility  of   the   writer   to  help  the  reader 
see.      Whenever   a   paper  divides   a   group   along   these  lines, 
with   some   readers   seeing   it   as    full   of   meaning  and  others 
believing   the  meaning  has   to   be   read   in,    it    is  usually 
best    to   abandon   the    sample   as  ^n   anchor,    thus,  **agreeing 
to  disagree."      In   this   case,   however,    the   expert  group 
reached   consensus,    choosing   to   regard   this   as   a   3  rather 
than   as   borderline.      The   consensus   did   reflect   the  ten- 
dency  of   this   group   to   score   a   paper   3   when    in  doubt. 

Novice  Teacher  Rater   Respons  e  >      This    group  was 
soiriewhat    inclined,    like^  myself,    to   score  "this  paper 
down,    not    only   for   brevity,    but    for  unreadability* 
They   responded  negatively,    as    I   had,    to   the  sentence: 
"I   had   a  hard   time   not    feeling   sorry   for   them  until  I 
was   the   one  manipulated  by   the  nurses." 
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SAMPLE  M 


Art  has  different  meanings  to  different  people.    Art  for  me  Isn't  what 
it  Is  for  another;  the  meaning  can  change  even  for  each  Individual  work 
simply  by  being  Interpreted  differently  in  the  mind  of  the  experiencer. 

M.C.  Escher  was  one  of  the  most  brilliant  and  talented  artist  of  our 
time.    He  died  in  1972,  but  his  works  live  on  with  the  image  of  his  mind 
forever  recorded  On  them.    To  look  at  one  of  his  works  (most  being  ink 
paintings)  is  to  look  into  a  strange  dimension  somehow  hidden  from  the 
rest  of  the  world,,, they're  alive  with  ideas,  every  one  of  them.  It 
was  the  dynamic  aspect  of  them  which  I  like  and  have  liked  ever  since  I 
first  saw  one  about  five  years  ago.    Many  find  his  painting  strange  and 
illogical,  not  pleasing  to  the  eye.    I  find  them  just  the  opposite,.  They 
contain  increadable  beaty  and  comfort,  both  of  which  I  saw  and'  felt  from 
the  start.    In  4^  world  of  hazard  and  discomfort,  Escher  is  a  refreshing 
escape  into  another  world,  one  of  those  escapes  we  all  need  from  time  to  time. 
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TEACHER  NOVICES 
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2 
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CONSENSUS  EVASION  (4)  EVASION  (3)  3/4  EVASION  (3) 

SCORE: 

X  "  first  choice  score,  no  count  made 

{}<)«  strong  second  choice  score,  no  count  made 


Discussion   o      Sample  M 

Researcher/Trainer  Response ,      A  problematic 
paper,    this   essay  represents   a   common   dilemma  for 
holistic   scorers.      At    first   glance,    I   read   it   as  an 
evasion   of   sorts.      If   I  had   encountered  it    on   a  College 
Board  English   Composition   test,    I  might   have   thought  it 
a   canned   essay,    a   favorite   collection   of  abstractions 
used   successfully  on   another  occasion   and  tailored 
slightly  to   appear   to   fit    the   topic.      However,    I  altered 
my  position   and   came   to   recognize   that    it    is  equally 
likely   that    this   paper   is   a   fairly  sophisticated  and 
ambitious   attempt   to   deal   vi t h   the   topic,    a  sincere 
response   to   the   Version   B   admonition   to  write   for  some- 
one  about   to  have   a   similar   experience.      Hence,  the 
writer^s   introductory   sentences   point   out    that  dif- 
ferent  people  have   different   responses   to   art.  Like 
a   few  others  who   responded   to  Version   B,    this  student 
interpreted   the   instructions*    quite  validly,    as  requir- 
ing writers   to   address   themselves   to   preparing  the 
reader   for   an   initial   experience,    rather   than  simply 
recounting  the  writer's   own   initial  experience. 
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scored   but    should   be   turned   over   to   the   trainer   or  the 
teacher.      This   is   done   to   avoid   giving   such  papers  mis- 
lea  d  i  n  g  1  y   lew  scores. 

Finally,  student   pointed   out    that    even   if  we 

thought   the   paper  was   on   the   subject,    it    should   get  a 
score   of   3  not   4   because   **it   would   be   a   better  paper  if 
the   person    really  had   told   us   something   about    his  first 
exper  ience---so  we   could   see   the   paintings.'*  Ultimately, 
this  was  the  view  that    I   took   of   the  paper. 

Expert   1  e  a  c  h  e  r  Rater  Response  .      The   experts  TJcjre 
adamant    that    this   paper  xvas   not    an   evasion,    and  scored 
it   with   little   disagreement    as   a   top  half  paper.  The 
group   split    50/50,    with   two   readers   giving   it    a  4  and 

giving   it    a   3.      Readers   were   not    able   to  reconcile 
this   difference;    staunch   defenders   of   the  maturity 
involved   in   what    the   student    attempted   and   the  quality 
of   his   abstractions   felt    that    this   paper   was    far  superior 
to   papers   such  as  A  and   H,    and   as   good   as   J,    although  dif- 
ferent   in  kind.      The   other   two   readers    agreed   with  the 
researcher /trainer   that    the   absence   of   supporting  details 
weakened   the   paper,    although   they   recognized   the   worth  of 
what    the   student   was   attempting   and   the    superior  quality 
of   the   abstractions,    including   the   introduction   and  con- 
k:  1  '.I  s  i  o  n  . 

N 0  V  i  c  e  Teacher   Rater   Respons  e  .      N (.) v i c e  teachers 
responded   to   this   paper  very  much   as   the    student  raters 
had:    they   admired    it    as    a   good   piece   of   writing,    on  the 
whole,    with   a   few  raters   giving   it    low   scores    for  fail-- 
ing   to   deal   with   the   topic.      Again   I    introduced  the 
option   of   labeling   a   paper   as    an   "evasion"   to  avoid 
using   a   low   score   for   a  well   written   paper.      As  with 
student    raters,    the   use   of   this   sample   paper   as  an 
example   of    an   evasion   did  not    cause   the    teacher  novices 
to   score   any   other   paper   this   way.      Strictly  speaking, 
then,    if   this   paper   is   not    considered   an   evasion,  then 
there   were   rro   evasions    in  the   entire   sample  of  student 
w  r  i  t  i  n  S:: . 


ERIC 


O  312 


With    this    in  mind,    a   main   shortcoming   of  the 
paper   is   the   lack   of   concrete   details   to   help   us  vis- 
ualize  what    it    is   Escher  does--in   this   case   an  enor- 
mously  difficult    task,    possibly  beyond   the  student's 
powers.      But   equally   possible   is    the   fact   that   in  thirty 
minutes,    the   student    found    insufficient    time   to    do  more 
than   work   out   the   fairly   strong  generalities   that  ex- 
press  his    feelings   about   Escher.      If  we   use  as  our 
rubric   the    standard   directions   to    teacher   scorers  at 
Sir   Francis   Drake   High   School   in  Marin  County,  Cali- 
fornia:   "Treat   each   paper  with  respect.      The  student 
should   be   rewarded   for  what    he  does   well,"  we  might 
find   ourselves   agreeing  with   the   one  expert   scorer  who 
awarded   this   paper   a   4.      I   decided    that   my  initial 
impression,    that    this   paper   was   an   evasion,   was  in 
error,    and    I  believe   that   with  a  more   careful  reading 
I   would  have   scored   this   paper   3.      The   claims,  albeit 
fascinating    ("they're   alive   with   ideas,    every   one  of 
them"),   need   some   attempt   at  substantiations-description 
either   of   the  paintings   or   of   one   or   two   of   the  ideas 
they   are   "alive   with."     However  difficult   this  might  be 
to   do,    the    student    set   himself   this   task  when  he  tackled 
this    subject,    and    from   the    rhetorical   perspective,  this 
piece   remains    unfinished.      The   implication   for  holisti- 
cally   scored   time  writing   tests   is,    of   course,    that  part 
of   the   student's   task   is   to    select   material   that    can  be 
turned   into   a   fairly   finished  piece   in  very   little  time. 
Some    subjects   are   too   big  and   too    important   to  be  dealt 
with   on   such   occasions.      And   yet,    it   will   be   the  most 
able   students   who    fall   into   this    trap,    a   fact  which 
mitigates   against    short    timed  tests. 

Student    Rater   Response .      I    took   an   informal  vote 
of   initial    responses    to   this   prompt   but    I   did   not  record 
them,    unfortunately,    since   I   expected   to   point   out  to 
students   that   this   paper   constituted   an   evasion.  In 
all    classes,    however,    I   observed   an   inverted   curve  in 
the   response;    that   is,    students  who   saw   the  paper  as 
fulfilling    the   topic    gave    it    one   of    the    top  scores 
(over  half    the   students   in   each   class   gave   this  essay 
a   4),    with   a   few   s t ud en t s '  g i v ing   it    the   lowest   score  as 
a  way   of   saying   that    it   did   not   satisfy   the  assignment. 
Comments   in   favor   of   the   top   score   for   this   paper  showed 
that    students   generally  were   impressed   and   attracted  by 
the    subject    chosen,    by   the    abstract    concepts,    and  by 
the   mature    voice.      When   comments   were   requested  from 
the    students   who    sc-red   this   essay   low,    the  response 
was    simply,    "It    doesn't   even   deal   with   the   topic!" ^  No 
students   were   able   to    show,    as   the   experts   later  did, 
how   the  paper   could   actually   be   read   as   very  much  on 
topic  ;    all    students   appeared   to   accept    the  trainer's 
initial   instruction  to   regard   this   paper   as  a  model  of 
evasion,    understanding   that    such  papers   should  not  be 
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SAMPLE  N 


The  first  time  I  waUed  through  an  airport  and  watched  the  planes 
lift  off  while  staring  through  the  Immense  plate  glass  windows  was 
really  exciting.    The  hustle  and  bustle  of  the  people,  the  noise  of  the 
jets  an  the  slick  floors  of  the  terminal  were  just  driving  me  crazy. 
There  were  so  many  different  types  of  people,  with  slanted  eyes.  Some 
people  couldn't  talk  correctly,  the  spoke  In  a  jibberlsh  sort  of  way. 
I  saw  more  types  of  paople  then  I  could  Imagine  while  I  was  sliding 
back  an  forth  across  the  slick  floors.    Mother  thought  they  were  dirty, 
but  but  she  was  wrong  they  weren*t  dirty  but  there  sure  were  aJot  of 
little  rolled  up  pieces  of  paper  with  lipstick  on  then,    the  best  was 
yet  to  come.   With  my  nose  pressed  tightly  to  the  window  I  could  see 
the  taxiing  around  the  runways.    I  could  even  see  a  few  taking  off, 
first  thf  front  and  then  the  back,  floating  through  the  air  as  easily 
as  1  could  Imagine. 

The  excitement  of  the  airport  Is  gone  now,  I've  seen  the  planes  take 
off  a  hundred  times,  and  I've  had  my  fill  of  slick  floors  and  cigarette 
butts,  and  the  people  who  were  different  I  now  see  every  day.    The  planes 
are  still  a  little  special  though  because  they  are  the  ticket  to  new  firsts. 
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CONSENSUS  3  2/3  3- 

SCORE : 


Discuss  ion   o  f   Samp  1  e  _N 

A  Note   f  rom  the  Researcher/Trainer.      Sample  N 
was   read  by  only   27   students,   having  been   Included  in 
the   first   t  raining  packet   as   the   last   paper   of   seven  > 
In   all   but   one   classroom,   discussion   of   the   other  six 
anchors   took  so   much   time  that    scoring  was   begun  without 
tabulating  responses   to   this   paper.      Considering  the 
spread  of  responses   from  other   scoring  groups,   I  regret 
not   having   recorded  more   student   responses.      I  found 
this   paper  parallel   in   general   quality   to   Sample  H,  the 
''mushroom"  paper,   and  wavered-^as   did   the   expert  raters- 
on  whether  to   award   it   a   2  or  a  3.      This   is  not  because 
I   don't   see   it   as  clearly  superior   to   certain  other 
papers   and   clearly  inferior  to   still  others,  but 
because   it    is  not   clear   to  me   where   the   cutoff  should 
come  between   2   and  3   for   this   sample   of   papers.  Like 
the   expert   raters,    I  have   read   the   California  High 
School  Proficiency  Exam   for  ETS   and  have  internalized 
a   set   of  expectations   that   leads  me   to   see   these  sam- 
ples   (N  and  H)   as  relatively  good  writing — strong  3's 
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on   a   5-point   scale.      But   the   scale   in   this   study  was 
only   four   points,    and   there  were  virtually  no  papers 
in   this   sample   equivalent    to   the  papers   that  earn 
scores   of   1  on   the   CHSPE.      This   would   tend   to  lower 
the   level   of  appers   such  as   this   one   to  a   score   of  2, 

Apart   from  the   debate   about   where   to   draw  the 
line   between   2   and   3,    the   paper   itself   creates  some 
debate.      I   read   the   paper   as   adopting   the  voice  and 
perspective  of   a   child   during   the   bulk  of   the  first 
paragraph,    then   returning   to   an   older   perspective  (and 
hence  voice)    in   the   final   paragraph,   with  strikingly 
effective   closure.      The  argument   between  readers  who 
placed   this   paper   in   the   upper   half   and   those  who  found 
it    typical   of   lower  half   papers   centered  on   wh ether  the 
childish    language   of    the  middle  passage    ("so   many  dif- 
ferent  types   of   people,   with   slanted   e y es . • . . Mo t he r 
thought    they  were  dirty  but    she  was   wrong   they  weren't 
dirty,,,")   was    indicative   of   an  undeveloped . writ ing 
style   or   was   an   attempt   to   assume  a   tone  appropriate 
to   the   child  who  was    sliding  around   on   the   slick  floors 
picking   up  cigarette   butts.      In   the   last   analysis,  all 
scorers   agreed    that    the  details   in   this   paper   and  the 
fine   ending  deserved   a  higher   score   than  papers  they 
had   agreed   to   call   2,    such  as   C,      It   might   be  useful 
to   consider  this   paper  typical   of   a   2/3   split  judgment. 
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BEYOND  THE  FIELD   TEST   OF  WRITING  PROMPTS: 
A  PRELIMINARY   INVESTIGATION   OF  THE 
NATURE  OF   RESPONSE   VARIATION  TO 
SELECTED   FIELD-TESTED  PROMPTS 

Gerald   Camp,    Charles   Kinzer,    and   Sandra  Murphy 


Part   I:        The   Effects   of  Assessment   Prompt  and 
Response   Variables   on   Holistic  Score; 
A  Pilot    Study   and   Validation   of  an 
Analysis  Technique 

Charles   Kinzer   and    Sandra  Murphy 


Part   II:      A  Field   Test    of   Eight    Prompts:  Santa 
Clara   County  Writing  Assessment* 

Gerald  Camp 


*A  Bay  Area  Writing  Project    Study   conducted  on 
behalf   of   the   Santa  Clara   County  AB  65  Consortium, 
which   provided   the   data   base   for   the    study  in 
Part  I. 
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Chapter   3:     Part  I 


THE   EFFECTS    OF  ASSESSMENT   PROMPT  AND  RESPONSE 
VARIABLES   ON  HOLISTIC   SCORE:      A  PILOT  STUDY 
AND   VALIDATION   OF  AN  ANALYSIS  TECHNIQUE 

Charles   Kinzer   and   Sandra  Murphy 

1 . 0  Introduction 

Holistic   scoring  procedures   for  writing  have  been 
widely   accepted  and  adopted  by   both  researchers  and 
practitioners   in   the   field.      Traditionally,  holistic 
scoring  has   been  used   to   rank  papers   for  placement  and 
competency   testing  purposes.      Following  such  rankings, 
some   researchers    (Cooper,    Cherry,   Gerber,  Fleischer, 
Copley,   and   Sartisky,    1980)   have   analyzed  characteristic 
features  of   the  written   product   in  the  holistically 
scored   sample.      However,    research   exploring  features 
of   the   assessment   writing   prompt   which  may  influence 
holistic   scores   is   a   relatively   new  phenomenon. 

Freedman    (1977),    in   an   examination   of   the  effects 
of   reader,    essay   and  environmental  variables   on  holistic 
scores,    found   that   essay  variables   contributed  most  to 
the   variance   in   the   scores.      Unexpectedly,    topic  effects 
were   also   found.      The   focus   of   the   study   reported  here 
is   upon   possible   relationships   between  assessment 
prompts,   written   products,    and  holistic  scores. 

As   part    of   a   larger  NIE  project,    this    study   is  one 
of   a   series   that   examines  writing  task  variables  in 
holistic  writing   assessments.      This   is   a  preliminary 
effort   to   determine  ways   of   identifying  effects  of 
prompt   and   response  variables   in   relation  to  holistic 
scores . 

This   study  therefore   had   two  major  goals: 

1.  To   investigate   prompt   effects   and  response 
variables  which  might   result    in  discrepancies 
in  holistic  scores. 

2.  To   validate   an   analysis   system   for  looking 
within   specific   prompts . 

A  fairly   rigorous   design  which  would  incorporate 
prompts,    scores   and  written  products  while  controlling 
for  extraneous   variables   is   shown   in  Figure  1.  Figure 
1   also   notes   the   basic   assumptions   upon  which  this 
study   is  based . 
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The   specific   design   for   the   study   parallels  that 
shown   in   Figure   1    (see   Figure   2),      The  modifications 
arise   from  this   study's   evolving   into   a   larger,  Bay 
Area  Writing  Project  assessment. 
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2 . 0     The  Data  Base 

In   1979,   the  Bay  Area  Writing  Project  and  the  Santa 
Clara  County  AB65  Secondary  Consortium  developed  a  set: 
of  writing  topics,   or  prompts,   which  were  to  become  part 
of  the   Consortium's  bank  of  evaluation   instruments.  The 
prompts  were  developed  by   a   subcommittee  made  up  of 
writing  teachers   from  Santa  Clara   County  elementary  and 
secondary  schools.      Prompts   from  several  sources  were 
examined  and  members  of  the  committee  also  contributed 
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topics   they  had  used   in  their  classrooms  or  districts. 
Twelve  prompts  were   initially   drafted   and  revised. 
(See-  Part    II   of   this   report   for   a   full   description  of 
the   data  base.) 

The   final   prompts   developed   by   the   committee  were 
field   tested   in   several   districts   throughout    the  country. 
Eight   prompts  were  tested. 

1.  Imagine   that   you   are   trapped   in  one   of  the 
following   situations   and   you   cannot   escape  for 
several  hours.      Tell  what   you   do   to   escape  or 
to   pass   the   time   until   you  are  rescued. 

a.    an   elevator  b.    an   abandoned  mine 

c.    a   locked   room         d.    a   life  boat 

2.  If  you  had  to  choose  to  be  something  other 
than  a  human  being,  what  would  you  choose? 
Give  your   reader   some   idea  of  what    it  would 

be   like   to   be   that   form,    and  why  you   chose  it.. 

3.  Think  of   a   personal   experience   that   has  in 
some  way   changed   your   life:    a   particular  event; 
a   person;    a  place  you  have  visited;    a  book  you 
have   read;    a   film,    television   show,    or  play 
you  have   seen.      Describe   the   experience  and 
explain  why   it   was   an   important   one    for  you. 

A.      Vhile  visiting   in  your   grandmother's  house  you 
find   a   large   trunk.      Propped   on   a   chair  beside 
the   trunk   is   a   card,    and   on   It   in-  your  grand- 
mother's  handwriting   are   the   words,    "To  my  only 
grandchild:    Inside  this   trunk  you  will  find 
what   you  have   always  wanted."     Describe  the 
gift    and   how  you   will   use  it. 

5.  Think  of   a   friend,    real   or   imaginary,    that  you 
had  when   you  were  younger.      Describe  something 
you   and   this   friend  did   together;   try   to  show 
your   reader   the   kind   of   person   this   friend  was 
and  why   you   chose   this   person   as   a  friend. 

6.  The   school  newspaper  has   asked   students   to  sub- 
mit  suggestions   about   how  to   improve   the  school 
Think  of   ONE   and   only   one   problem  which  you 
would   like   to   see   solved   to  make   life   at  school 
better   for  you   and   others.      In   a   letter   to  the 
school  newspaper   describe   the   problem   and  tell 
how  you  would   like   to   see   it  solved. 

7.  Employment    agencies   can   be  very  helpful  in 
finding  you   a  job   if   they  know  what  job  you 
want   and   are  qualified   for.      Write   a  letter 
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8,      You  have   been   sent   a   bill   charging  you    for  two 
record   albums  you   did   not   purchase.      Write  a 
letter   explaining   the   problem  so   th^t  someone 
m   the   billing  department   can   correct    the  error. 

The   bill   is    from  Melodie   Record  Club, 

Tune   Street,    Nashville,    Tenn.  78056 

The   records   charged   to   your   account  are: 
Kiss  Presents   Opera  Highlights  and 
Led   Zepplin   Count  ry  Hits 

Your  name  and  address  are: 

Chris  Brown 

456  Main  Street 

Bay   City,    CA  92001 

Your   account   number  is: 

35-800-21 
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tile   ranking  on   CTBS,    and   the   county-vide   proportion  of 
schools  which   fell   into   each  category  was  determined. 

The   mid-range   sample    formed  the   data  base    for  the 
study   reported   here.      In   this   sample   eight  classes, 
divided   among   three   schools,   were   tested.      One  school 
fell   into   the   34-44   percentile   range,    one   into   the  45-55 
percentile    range,    and   one    into   the   56-66  percentile 
range.      These   schools  were    further   categorized   as  low, 
medium,    and  high   respectively.      The   eight   prompts  were 
tested   in   one   **low,''   two   "average"   and   one  "high" 
ability  class. 

In   the   three   mid-range   schools,    between   23   and  30 
papers  were   obtained    from   each   class.      The   eight  prompts 
were   randomly   assigned   to   the   students   in   each  class. 
Testing  was   done   during   a    five-day   period.  Students 
were   given    30  minutes   to  write   in  response   to   the  given 
prompt.      The   data   from  the   above   field   test   of  prompts 
were   made   available   to   the    investigators   for   the  study 
reported   here.      As   noted,    identification   of   the  effects 
of   the   prompts,    and   the   dc^yelopment   of   systems  for 
investigating  and   analyzing   such  effects   are  the  ma j  o  r 
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goals   of   this   study.      In  keeping  with   the  prospective 
design  noted   in   Figure   1,    two  prompts   were  considered 
ideal   for   the  purposes   of    this   study.      Mean  scores 
between   prompts   5   and   6  were   significantly  different. 
Such  a   difference  was   unexpected   due   to   the  randomiza- 
tion  in   assignment   of   prompts   and   controls   in  the 
scoring   procedures.      Thus,    the   papers   resulting  from 
responses   to   these   two  prompts   were   selected   for  analy- 
sis   in   this    study.      In   short,    prompts    5   and    6  were 
selected   because   they   showed   the   widest   variation  in 
average   holistic   score.      Means   and  variances    for  the 
two    topics    are    shown    in   Table  1.* 

The    sub-sample   selected    for    analysis   was    drawn  from 
the   entire  pool   of   responses   to   prompts   5   and   6.  To 
ensure   that   the   sub-sample  was   representative   of  the 
total    data   base,    the    data   base  was    divided    into  number 
of  papers   per   score.      Within  each   score,   papers  were 
randomly   selected   so   that    overall   percentages   of  papers 
per   score  were   equivalent    across   the   total   response  set 
and   the   sample   to  be   analyzed.      For   example,    if   20%  of 
the   total   data   set   received   a   score   of   8,    then  the 
sample  "as   selected   so   that   20%   of   the   total   to  be 
analyzed   received   a   score   of   8.      In   this  way,  the 
sample   was   both   randomly   selected    and  representative 
of   the   data   set,    allowing   generalization   to   the  popu- 
lation  from  which   the   original   data  was   gathered.  Forty 
papers,    twenty    each    from    the   set    of    responses    to  prompts 
5   and   6,   were   selected   for   analysi-s    (see  Table  2). 


3 . 0     The  Analysis  System 

After   the   sample  was   selected,    a   task  analysis 
system  was   developed   to   provide   a  means   for  identifying 
the   salient   aspects   of   the   prompts   and   s  t  u  d  en  t  "r  e  sp  on  s  e  si , 
The  prompt-related   task  analysis   system  was   based  specif- 
ically  on   the   two   prompts.      The   analysis   system  should 
be   viewed   as   an   exploratory  methodology.      It   was  con- 
structed   to   allow  examination   of   performance   and  prompt 
factors   in   relation   to  holistic    score   bofh  within  and 
across   prompts.      Global   analysis   variables   in  the 
response   analysis   included   legibility,    general  fluency 
in    terms    of   the   amount    of   writing   produced,    use  of 
cohesive  devices   in   the   development    of   ideas,  and 
errors   related   to   grammatical   and  written  language 
conventions. 


3 • 1      Response-based  Variables 

3.1.1Legibility.      A  measure   of   legibility  was 
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obtained   through   an   analysis   procedure  previously 
er»ployed   by   Keech   and   Blickhahn    (1981).  Individual 
papers   were   ranked   on   a   scale   of   1   to   A   by   two   of  the 
investigators.      Factors   used   for   the   legibility  rankings 
are   shown   in  Table   3.      Each  legibility   score  was  based 
\ pon   two   independent  rankings. 

3.1 .2    Cohesion.      A  measure   of   idea   development  was 
obtained   through   the   use   of  an   adaptation  of  Halliday 
and   Hasan's    technique   for   analyzing   cohesion   in  English 
(Halliday   and   Hasan,    1976).      When   groups   of  words  within 
a   text   were   related   to   the   same   idea,    they  were  treated 
as    textual   "chains^'   of   ideas.      As   in  Halliday   and  Hasan's 
system   of   analysis,    these  words   or  tokens   could  be  repe-- 
titions    of   the   same,  idea,    synonyms,   or   reference  items 
which  presuppose   the   same   meaning.      The   basic  assumption 
underlying   this   portion   of   the   analysis   was   that  repeated 
reference   resulted   from   the  development    of   ideas  which 
constituted   central   themes   in   the  texts. 

In   each   paper,    the  number   of   cohesive   ties   in  the 
three   longest   text    chains   was   counted.      Lexical  ties 
and   other   cohesive   ties   produced   by  substitution, 
reference   and   ellipsis   were  included  in   calculating  the 
length   of   the   text    chains.      Conjunctive   ties   were  counted 
separately    from  other  types   of   ties,    since   such   ties  are 
often   used   to   specify   the   relationships   between  more  than 
one   idea   or  -topic.      Conjunctive   ties   were   tabluated  as 
addit  ive    (e.g.,    and,    and   also,    nor,   neither,    or,    or  else, 
and. ..not,    alternatively,    etc.),    adversative    (e.g.,  yet, 
but,   however,    although,    nevertheless,    still,  etc.), 
causal    (e.g.,    so,    the,    thus,    therefore,    since,  because 
of,    etc.),    or   temporal    (e.g.,   next,   before,    then,  after, 
subsequently,    this    time,  etc.). 

3  .1.3   Errors  .      Transcript  ion   errors,  grammatical 
errors,    and   errors   related   to   standards   of  usage  for 
written   language  were   also   counted   in   each   paper  of  the 
sample.      The   errors   were   classified  into   12  categories: 

1.  Unattached   dependent   clauses    (e.g.,    "When  he 
was    fifteen."),    and   structures   which   lacked  a 
complete   verb    (e.g.,    "The  boy   running  down  the 
hill."),    were   counted   as   fragments . 

2.  Comma   splices  vere   counted   if   a   comma  was  used 
to  join   two   or  more  independent    clauses  when 
there   should  have  been   two   sentences  separated 
by   a   period   or   a   semi-colon    (e.g.,    "I   chose  to 
stay   because   of   the   fact   that   some   people  are 
different    from  others,    they  have   their  own 
perceptions  and..."). 

3.  Comma  omissions   were   counted  when   commas  were 
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missing  after   initial   introductory  clauses, 
etc.    (e.g.,    "When  the   hoy   came    running  we 
saw...").      They  were   not    counted  when  they 
were   ctr.itted   around   phrases   in    the  second 
clause   of   sentences   that    c  ont  a  ined   two  coor- 
dinated clauses. 

Sub.i  ect-verb   agreement    errors   were   counted  w!jen 
subjects   and   verbs   did   not   agree   in  number 
(e.g.,    "The   fastest    boy   and  the   fastest  girl 
runs    tomorrow. ") . 

Non-standard   verb   usage    errors    included  items 
as   "I    seen...,"   "He   brung...,"    items  related 
to   dialect   differences,    or   items  inappropriate 
for   written  texts. 

Pronoun   agreement   errors   were    counted  when 
pronouns   did   not   agree    in  number  with  their 
referents    (e.g.,    "The   funny  thing  about  the 
boy  was   that    they   like   to  swim."). 

Non-standard   pronoun   usage   errors  included 
items   such   as    "Sally   and  me  went    to  the 
party...,"   "Us   boys   went    to   the  beach- 
"He   went,  with   John   and   I,"  etc. 

Ambiguous   reference   errors   included  instances 
^ere   a   pronoun   could   be    interpreted   as  refer- 
ring  to  more    than   one   person   or   thing  mentioned 
in   the   text    (e.g.,    "Bobby   smiled   at    Tim  when  he 
won."),    as  well   as    instances   where   no  antecedent 
appeared   in   the   text   preceding   the    use   of  a 
pronoun . 

Modification    errors    included   misplaced  modifiers 
"(e.g.  ,    "There   was   a   person   on    the   bus   with  a 
brown   cap."),    two-^way  modifiers    (e.g.,  "The 
mavor   said   in   his    speech  his    brother  Xold  the 
truth."),    and    dangling  modifiers    (e.g.,  "While 
correcting  papers,    the   message   came    from  the 
principal.") * 

Misuse   of   adi  ectives   and   adverb   errors  included 
Adverbs   used    as   adjectives    (e.g.,    "The  carpen- 
ters  built    the   platform   sturdily   enough.  ), 
and   double   conipar  i  sons    (e.g.,    "more  happier," 
"most    farthest  ,"   etc  .  )  . 


S p e  1 1  i n_g  errors  . 

The  final  category  was  used  for  tabluatirg 
errors  other  than  those  noted  above.  Item 
which   appeared   in   this    category  include^. 

-^j  1 
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f  r  r  r  s  s u ^  h  a  s  d u b  1 n  e  a  t  i  v s  ,  and  v ^ ■  r  J  ;^  u s  f  d 
in   a  ;■•  r  r  o r  i  a  t  e   c  c>  ri  t  e  x  t  >r> . 


^ .        P  r/-j!ij>  t      /i  B  e  d   V  aria  b  1.  e_s 

The    prL  rp-'t -based   analvsis   svstem  was    devised  to 
spf.ify   task   demands    in   the   prompt.      These  variables 
c  >uld   be   exar^ined   in   relation   to   the   response  based 
variables    f'.'und    in    the   student    papers.      Student  papers 
were    independently    read   by   two   investigators  who  decided 
wrietb.er   '-r  not    the   response   iriatched   the   category.  Areas 
of    d  i  s  a    r  e  er.en  t   were   res'olved   bv   consensus   or   by   a  third 
1-  a  r  r  V  arbiter. 

The   three   categories   of    proTTipt-based   variables  were 
s  '.■  r  ■■■  L  I  i    d   a  s   be  i n  1^  : 

A  .      D  i  r  e  c:  t  I  V   related   to   the   e  x  p  ].  i  c  i  t  1  y   stated  task 

d  e r, a d  s    in    the   p  r  o rr. p  t  h  ; 

h  .      E  1  a  b    r  a  t  i  o  n  s   of   e  x  p  j.  i  c  i  t  1  v   stated   task   d  e  ni  a  n  d  s 
1  n    the   p  r  -    p  t  s  r 

■ «      Divers^ent^    i.e.^    containing   issues   or  themes 
not    called   ftor   in   the   prompt   or   which  contra- 
dicted  e  X  p  I  i  r:  i  t  1  y   stated   task  d  e  iri  a  n  d  s  . 

0'  .2  . 1    P  r_D_irj  t  -  has  e  d   Va  r  iab  1  e  s    fo_r  Pr__o__npt    5  .      P  r  o  n  p  t: 


Think  of   a   friend,    real   or  iniaginary,    that    you  had 
when   you   were   younger.      Describe   something   you  and 
tbiis   friend   did   together;    try   to   shov;  vour  reader 
t?iH   kind   of   person   this    friend   was   and  why  you 
-h^--se   this   pers^-^n   as   a  friend. 

'ihe   derotnds   of   this   proript,    as   they   were  expected 
.    r  t  .*^M  e  1-  t  e  d    in   the   strident   responses,  were: 

Category   o  f   Prompt    penan  d 
(A= explicitly  stated; 
h  ^T:i-I--  B=^elab  oration;    C  =  divergent) 

1  .      S  p  fc    1  f  i  e  d   a    i  r  i  e  n  -A  B 
r  e  1  a  t  i  o  n  h  h  i  p  .    (  r^e  a  ! 
friend) 

.      E  e  zi  V    in   r  e  c  e  r;  t   p  a  s  t  .  A 
f  P  -  s  t    u  I  enun  t  a  r  V   s  c  h o  ^'M  ) 


Event  in  distant  past, 
t^pre   secc^ndary  school) 
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Category  o  f   P  romp t  Demand 
(A= explicitly  stated; 
Prcr.j't   DeTi -^d  B  =  e  1  ab  o  r  a  t  i  on  ;  C=divergent) 

4.  escribed   cne   event.  A 

5.  Pescribt^s   related   events.  B 

6.  Lists    separate   events.  C 


7  .  E  X  p  1  i  c  i  t  1 discusses 
character  traits  and 
personality. 

B.      Implicitly    shows  character 
through   event . 

9.      Implicitly   shows  character 
through   event.  (i.e., 
comments   on  relationship 
betwee.    character   and  event) 

10.  Explicitly   discusses  why 
person   was  chosen. 

11.  Implicitly   shows   xv^hy  person 
waft   c:h.">sen   through  event. 

1.-:.      Explicitly   shows   why  person 
t  h  r  C'  u  g  h  event.  (i.e., 
riommients   on  relationship 
between   event   and  why 

p  e  r  s  '^'^  n   was    c  h  o  s  e  n  ) 

1  .  .      D  e  s  ■/  r  1  b  e  s   a    friend.      f  1  .  e  .  5 

p  li  V  s  i  c  a  1   c  h  a  r  a  c  t  e  r  i  s  t  1  c  s  ) 

I',.      -;pe::ifies    friend  relation-- 
si:  i  p  .       (  it:,  a  fj;  ina  r  v  friend) 


Im.pl  ic  it  Iv   shows   character  (' 
traits   and   personal  it  v. 

Pr^:mpt-baj^e_d  VariaM„es  ^ill  Prempl  6.  Promrt 


The    schO'^1   newspaper   has   asked   students  to  submit 
suTriestions   about  how  to   improve   the  school. 
Think   .>f   ONE   and   onlv   one  problem  which  you  would 
like   t-   see   solved   to  make   life   at   school  better 
f.  r   V..U   and   others.      In   a   letter   to   the  school 
newspaper   describe   the   problem   and   tell  how  you 
w .  •  u  1  vi    like   to   see   it  solved. 
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The   demands   of   this    proinpt,    as   they   were  expected 
be   reflected    in   the   student    responses,  were: 

Category    o  f   Pr omp  t  Demand 
(A= explicitly  stated; 
Prompt   Demand  B=^elaborat  ion  ;    C=  d  i  ve  r  gen  t  ) 

1.  Refers    to    request    for  B 
suggest  ions . 

2.  Explicitly   c^r   obviously  B 
relates   problem   to  school 
improvement . 


3.      Addresses   one   problem/  A 
issue. 

A.      Addresses    related  B 
problem/ issues . 

5  -      Lists   sepa  rate   p  r  o b  1  e m> s  ,/'  C 
i  s  s  u  e  s  . 

.      Relates   problem   to   at  h  e  r  B 
a  s  p-  e  c:  t  s    of    s  c  h    o  1    i  i  f  e  . 

Explicitly   states   solution  A 
vould   make   things  better 
for  self. 

^ .      Explicitly   states   solution  A 
would   make   things  better 
for   others . 

9,      Explicitly   states   solution  B 
would  make   things  better 
^        everyone    (i.e.,  staff, 
and   st  udent  s ) . 

H?.      Letter   in   form.  A 

11.  Letter   in   fact    (i.e.,  A 
first  person). 

12.  Describes   the   problem.  A 

13.  I'ixplains   why   it's   a   problem.  B 

14.  Explains   why   the   problem   is  B 
significant    (i.e.,    in  con- 
par  ±  son   to   other  problems). 

15.  Presentssolution.  A 
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Ca  t  e  gory   o  f   Pr omp  t  Demand 
(A= explicitly  stated; 
'  Pror.rt    D  e   a  n  d  B=  e  1  abor  a  t  i  on  ;  C=divergent) 

16.  Discusses   effects   of  B 
s  o  1  u  t  i  c  n  . 

17.  Explains   worth   of  B 
sc»lution    (i.e.,  in 

relation   to   other  issues) 

18.  Implicitly   describes   the  C 
problem. 


4 . 0     Data   Tabulat  ion   an  d  Analysis 

Individual   papers   were   coded   on   a  yes/no   basis  for 
each   of   the    items    in   each   category.      The   papers  were 
independently   coded   by   two  members    of   the    research  staff. 
There  was   89%   agreement   between   the   two   coders   on  the 
items    for   prompt    5,    and   91%   agreement   on    codings  for 
prompt    6.      On   items   where   codings    disagreed,    the  papers 
were   discussed   and   a   consensus   was    reached.      On  one 
student   paper,    there  was   disagreement   on   three   of  the 
items.      This    paper   was   eliminated    from   the  analysis. 
Thus,    the   sub-sample   for   topic   6  was   reduced   to   an  N 
of  19. 

When    looking   at    the    above-noted   task   demand  items 
for   prompts    5   and   6,    it    appears   that   some   of   the  task 
demands   are    comparable   across   both   prompts.      For  example, 
both  prompts    call    for   the   s t uden t - wr i t e r    to   focus   on  a 
single    subject.      If    the   writer   did    in   fact    focus   on  one 
subject,    an   expectation   which  was    in   the    topic  demand 
category   ''A"    (i.e.,    "describes   one   event"   was  explicitly 
stated   in   the   prompt),    the    student    response  was  coded 
"yes,"    i.eo,    as   having  met    the    explicit    task  demand. 
If,    on   the   other   hand,    the   student   wrote   about  several 
unrelated    events    and   did   not    focus    on   a    single  subject, 
then   that    item  would   be    coded   "no."     However,  this 
latter   response  would   be   coded   "yes"  with   regard    to  the 
item  "lists    separate   events"    (an   item  in    category  "C": 
divergent    response),    since   writing   about    several  unre- 
lated  events    or   problems   contradicts   the  prompt's 
ins  t  rue  t  ion  s . 

A   student    could   also  write   about   more   than  one 
event    or   problem,    relating   them   to   a   central   theme.  In 
response   to   prompt    6,    for   example,    one    student  argued 
that    students   be   permitted   to   enroll   in   seven  classes, 
*rather   than   the   allowable   six,  so  that   they   could  fit 
electives   into   their   schedule.      In    constructing  his 
argument,    the   student   discussed  various   problems  which 
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were   related   to   flexibility    in   scheduling  and   time  for 
elective   classes.      Another    studenc    wrote   about  the 
gene'ral   problem   of   vandalism,    but    elaborated   on  this 
theme   by   citing   several    specific   problems   as  examples 
and   then   suggesting   solutions   for   each.      In   response  to 
prompt    5,    one    student    writer    interwove    co  mm  ents   about  a 
soccer   try  out   and   the   publishing^  of   a   play    (two  separate 
events),    in   order   to   show   two   sides   of   her  friend's 
character.      Responses    sur^.i   as   the    above   were    coded  (in 
cate^sory  B)    as   elaborating   on   demands  specifically 
called    for   by   the  prompt. 


3 . 0     Result  s 

Results   were   analyzed   both  within   and   across  prompts 
The   within-prompt    analysis    provided    information  about 
interactions    between   the   writer   and   the    prompt,    by  sug- 
gesting which   parts    of    the    prompt    were    focused   upon  by 
different   writers.      Obviously,    some   aspects    of   a  prompt 
are   of   more    importance    to    a   writer   than    other  aspects. 
The  within-prompt   analysis    is   a   preliminary   attempt  to 
identify   implicit   and   explicit   prompt    demands   which  are 
Trc:st    salient    to   the  writer. 

Traditionally,    researchers   who    examine    effects  of 
prompts    do    so   only    in    relation    to    score.      However,  some 
factors  may   hold   constant    across   prompts.      Ac r o s s - p r omp t 
analyses   in    this    study   examine   the    following  variables 
in   an   effort    to   explore   such   possible  constants: 


1 . 

total   number   of  wor 

ds    in    the    re  spons 

e ; 

legibility ; 

3. 

proportion   cf  total 

errors   to  total 

numbe 

r  of 

wor  d  s  ; 

. 

proportion   of  numbe 

r   of   words    in  the 

thr  e 

e 

longest    text  chains 

to    total   n  umbe  r 

0  f  wo 

r  d  s  ; 

5  . 

proportion   of  words 

in   the    Ion  gest  t 

ex  t  c 

ha  in 

to    total   number  of 

wo  r  d  s  . 

Table  A 

presents    the  results 

of    the  analyses 

p  e  r  f  0 

r  med 

on  each 

of   the   above  variabl 

es   across   both  pr 

omp  t  s 

As 

shown   in   Table    4,  on 

ly   proportion  of 

total 

errors 

was   not  statistically 

significant  aero 

s  s  pr 

omp  t  s 

This   indicates   that  students 

made   the   same  nu 

mber 

of 

errors 

regardless   of   which  p 

rompt    they   were  g 

i  ven . 

The 

other   four  variables,  howeve 

r,   we  re   found  to 

dif  f  e 

r 

across 

prompts.      In   terms  of 

statistical  sign 

ificance. 

students  wrote  more  words  in 

responding   to  pr 

omp  t 

5  ,  as 
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veil    as  writing   Ic^r^er   text   chains   and   including  more 
words   in   their   three    longest   chains,    than   did   the  prompt 
6   re'spondent  s .      The   pron:pt   6   papers  were    found   to  be 
significantly  ir.ore   legible   than   the   papers  written  in 
response   to  proir.pt  5. 

The    four   statistically   significant    variables  noted 
above  were   exar.ined   in   relation   to   score.      The  results 
of   this   analysis   are   shown   in   Table  5. 

Table   5   clearly   shows   a   significant  relationship 
between   prompt    6   and   the    four  variables    analyzed.  In 
prompt    5,    no   statistically   significant  relationship 
between    score   and   any  of   the   four   factors  was  found. 
In   prompt    6,    huwever,    all   of   the   variables  were  found 
to  have   a   statistically   significant    relationship  to 
score.      As   score   in   prompt    6   increased,    so   did  legi- 
bility  rank.      However,    as   score    increased, 'proportions 
(with   regard   to   total   number  of   words   in   the  response) 
of   errors,   number   of   words    in   the    longest   three  text 
chains,    and  number   of   words   in   the   longest  chain 
decreased.      These   results   are   elaborated   upon  in 
Section  6.0. 

As   noted   in   Section   3.2,    the   task   analysis  system 
was   constructed   to   include   three   categories   of  resppnse- 
reflected   task   demands.      The    first    category  C'A") 
explicitly   asked   for   information   to  be    included    in  the 
response    ( e  .  g  .  ,  ' " t h ink   of   ONE   and   only   one  problem"). 
The   second   category    ("B")    implicitly  asked   for  informa- 
tion  to   be   included   in   the   response,    thus  inviting 
elaboration   by   the  writer.      For   example,  "Describe 
something  you   and   this    friend   did   together"  implies 
a   past    event   which  was    a   "one-shot"   affair,   not  an 
ongoing   activity.      The    third   category    ("C")  analyzed 
response    characteristics  which   included  information 
that   was   not    called   for   or  was   inappropriate    to  the 
prompt.      For   example,    the    student   may  have  listed 
several   events    or   problems  when   one  was    clearly  asked 
for.      These   three   categories  were    compared   to  score 
both   within   and   across   the    two  prompts. 

Table   6   presents    results    of   the   analysis   used  to 
det  ermine   p'^ossible   relationships   between   score  and 
explicit,    elaborated   and   inappropriate  information 
found   in   the   written   responses.      As   noted   in  Table  6, 
there   is   no   statistically   significant  relationship 
between    score   and   any   of   the   three   independent  measures 
in   prompt    5.      In   prompt    6,    however,    results  indicate 
that   as    score   increases,    so   do   instances   of  elaborated 
response.      Further,    as    scores    in   prompt    6  increase, 
inappropriate    response    interpretations    decrease.  This 
Is   of    special    interest   when   it    is   remembered   that  the 
mean   score   for   prompt    6  was   higher   than   that  J^^^-'^rompt 
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5,  and    is   a   possible   result    of   prorr.pt    effects  rather 
than   wr i t e r- r e 1  a t e d  factors. 

The   above   result    is   paralleled  by    findings  noted 
in  Table   7.      A  t-^test    on   the   means   across  prompts  for 
the   three   response   categories    shows:    (1)    that  signifi- 
cantly more   students   responded   to   explicitly   asked  for 
information    in   prompt    6;    (2)    that    there  was   no  difference 
between   prompts    in   number   of   elaborated   responses;  and, 
(3)    that    significantly  more   responses   in   prompt   5  fell 
into    the    inappropriate   response    category   than    in  prompt 

6.  These    results   also   appear   to   support   the  contention 
chat    there    is   an   effect    of   a   writing   prompt   which  is 
manifested   by   differences   in   responses    and   by  holistic 
score   between   these    two  prompts. 
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6 ■ 0     Discussion  ^ 

The   results    support    the   viewpoint    that    effects  of 
writing  prompts    are   related    in   student    responses  and 
influence   holistic   scores.      Since   the^writing  assessment 
distributed   the   prompts   randomly   across    schools  and 
students,    no   difference    in    group   mean   scores  should 
result.      That    is,    the    randomization   procedure  ensures 
equivalent    abilities    of   students    involved   in   the  writing 
task,    regardless   of   prompt.      Yet,    there   were  differences 
in  mean   holistic    scores    found   across   responses   to  prompts 
5   and  6. 

The    differences    in   scores   across   prompts    could  be 
attributed   to   at    least   two    factors.      First,    one   of  the 
prompts   may   be  more   difficult    than   the   other.      One  way 
to   examine   this    possibility   is   to   see   if   writers   can  do 
what    the   prompt    calls    for.      Such   a   technique   was  employed 
in   this   study.      Second,    differences   in   score  across 
prompts   could   result   because   the   readers    (i.e.,  the 
holistic   scorers)    respond   differently   to  students' 
writing  based   upon   external,    non-prompt    related  factors 
such   as    legibility   and  number  of   errors.      This  aspect 
was   also   explored   in   this  study. 

The   analyses   employed   indicated   that   students  did 
more   of   the    things   called    for   in   prompt    6   than   in  prompt 
5.      Prompt    6  was    also   the   one   that   had   the   higher  group 
mean   score.      Thus,    it    seems   appropriate   to   say   that  the 
more    implicit    and   explicit    demands   in   a   prompt"    that  are 
addressed,    the   higher   the    score.      In   short,    the  analysis 
system   indicates    that    prompt    5's    task   demands   were  not 
well    identified   and   responded   to,    implying   that  prompt 
5  was   more  difficult. 

It    should   not,    however,    come   as   a   surprise  that 
prompt    5  was  more   difficult    than   prompt    6   in   terms  of 
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task   denands.      Prcrr.pt    6   asked    the   students   to   think  of 
a   school-related   problem,    explain   why   it   was    a  problem, 
and  'then   suggest    a    solution.      Prompt    5   asked  students 
to   think  of    a   friend,    then   to    remember   and   describe  an 
event    involving  both   the    student-author   and    the  friend* 
The    event    was    then    to   be    used    to    illustrate    the  friend's 
character   traits,    in   addition   to   showing    (through  the 
event)    why    the   person   was    chosen    as    a    friend.      It  seems 
clear   that    the   task   demands   for   prompt    5  were  more  dif- 
ficult   than    those    of  prompt   6,    and   that    students  could 
Tr.">re   easily   meet    the   task   demands    in   prompt    6.      The  fact 
that    this  was    illustrated   through   the   analysis  system 
used    in   this    study   supports   the   use   of   such    analyses  as 
a  method   for   examining   prompt    difficulty   based  upon  task 
demands. 

The    finding,  with   regard    to   errors    and    text  chains, 
however,    is    somewhat    surprising-      It   was   expected  that 
chain    length    (measured    here   in   number   of   "chained'*  words 
per   total   number    of   words    in    the   written   product)  wculd 
parallel   increases   in   score,    rather   than   decrease  as 
implied   by    the   results.      This    finding,    however,  could 
be   due   to'  an    artifact   of   the    length   of   the  student's 
paper.      For   example,    as   response    length   increases,  the 
propor t  ion    of    chained  words   to    total   words  decreases 
even  though  the  number   of   words    in   a   chain  may  be  sub- 
stantial.     If   a   response    consisted    of    100   words,  with 
a   longest    chain   of   5  words,    the   proportion   is    .05.  If 
the   response    is   125   words   long,   with   chain  length 
remaining    constant,    the    proportion    is    lower  (.04). 
Yet    one   cannot   measure   chaining   purely   on  ^the   basis  of 
number   of   words   in   a   chain   since   this    could   be   a  func- 
tion  of   length.      Thus,    even   though   limitations  exist, 
the    proportion  measure    is    useful   when   one   wishes  to 
look   at   a   text   as    an   entity,    with   chains   as    a  part  of 
the   text,    rather   than   the   chain   as    an   entity  removed 
fromi   the  text. 

It    is    probable,    however,    that    elaboration   of  an 
idea  within    a   chain   takes   place   as   number   of   words  per 
chain    increases.      It    is    this    elaboration,  therefore, 
which   may   be    felated   to   an   increase   in    score,   but  the 
necessary    increase    in    length    of   written   response  which 
results   from   this   elaboration    also   results    in   a  lower 
proportion   measure.      This   explanation   could   also  account 
for   the  negative    and   significant    correlation   noted  in 
Table  5. 

The   study   also   examined   variables   which  remain 
constant    across   prompts .      Results    indicated    that  students 
made    the   same  number   of   errors   responding   to   each  prompt. 
This    further    supports    the   view   that    students    across  the 
two   prompts   were    of   equal   ability.      Legibility   and  length 
of   texts,    however,    were    significantly   different  when 
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compared   across   topics.      Prompt   6   texts   were   both  more 
legible   and    longer   than    texts    from  prompt    5.      The  within 
promT)t    analyses,    however,    were   ambiguous    in   these  areas. 
Prompt    6   results   showed    that    legibility   increased  as 
score    increased.      However,    length   of    response   was  not 
significantly   related   to   score    in   either   topic,    nor  wa 
legibility   related    to   score   in   topic   5.      It    is  indicat 
that   within   prompt   variables    (i.e.,    task   demands)  have 
more   influence   on   score   than   do   external  variables  such 
as  legibility. 

In   c on  elusion,    this    study   has  met    its   goals   as  a 
pilot   project.      It    is   clear   that    task   demands   can  be 
related    to   score,    and   that    an   analysis   system  modeled 
on   the   one   employed   herein   can   be   valuable   in  determining 
how  well   stud  en ts   respond    to   a   prompt,    and   in  determining 
how  difficult    a   prompt    is.      On   the   basis   of   these  pre- 
liminary  results,    further    research    in    the    area   is  justi- 
fied  and  recommended. 

The   refinement    of   procedures    for    the  identification 
and   analysis    of   effects   of   writing   prompts   is   a  necessary 
step   toward   devising   ways   to   help   teachers   and  evaluators 
improve    the   formulation   of   prompts    for   writing  assessment 
and    instruction.      Given    the   widespread    use    of  direct 
measures    for    evaluating   writing   competence,    every  effort 
should   be   made    to    control    for   unintended    effects    in  the 
prompts   used    in   these  assessments. 
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Table  1 


Means  and  Variances  for  Prompts  5  and  6 


N 

X 

SD 

Prompt  5 

91 

8.67 

3.13 

Prompt  6 

76 

9.46 

3.26 
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Table  2 

Distribution  of  Papers,  by  Prompt  and  Score, 
Randonily  Selected  for  Analysis 
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Table  3 
Legibility  Ranking  Scale* 


Legibility  Rank  Criterion 


Clear  and  legible,  even  if  not  attractive*  Margins 
do  not  interfere  with  reading. 

Relatively  hard  to  read.  Ind'i^ts  too  far,  distracts 
reader. 

Very  hard  to  read,  some  words  illegible.  Omits  left 
and  right  margins,  writing  extends  to  edges  and  dis- 
tracts reader. 

Illegible,  wome  parts  impossible  to  read.  Random 
line-skipping  and  indentation  which  detracts  from 
comprehension. 


*  TAM  Skills  Test  —  Writing  Feature  Analysis  (Keech  &  Blickhahn,  1981). 


i 
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i 
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Table  4 


Aualvfis  -ji  -Tctal  Kords,  Legibility,  Proportion  of  Total 
Chain  Vord^,  Proportr.on  of  Largest  Chain  Words  and 
Prc^portion  ft  Total  Errors  Across  Prompts  5  and  6 


Variable  N  X  SD  df 


Total  Words  (topic  5)  20  222.35  81.48 

Tocal  Words  (topic  6)  19  157.57  40.97 

Legibility  (topic  5)  20  2.80  0.62 

Legibility  (topic  6)  19  3.16  0.38 


Proportion  of  Total  Words  in 
Longest  3  Chains  to  Total  Words 

(topic  5)  20  0,16  0.03 

Proportion  of  Total  Words  in 
Longest  3  Chains  to  Total  Words 

(topic  6)  19  0.12  0.03 


Proportion  of  Words  in  Longest 

Chain  to  Total  Words  (topic  5)  20  0.08  0."02 
Proportion  of  Words  in  Longest 

Chain  to  Total  Words  (topic  6)  19           0.05  0.02 


Proportion  of  Total  Errors  to 

Total  Words  (topic  5)  20  0.07  0.03 

Proportion  of  Total  Errors  to 

Total  Words  (topic  6)  19  0o06  0o04 


*  significant  at  p^O.05 


37  3.11* 


37  -2.18* 


37  4.03* 


37  4.90* 


37  0.16 
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Table  5 

Across  Prompt  Analysis 
(Kendall's  Correlation  Co-efficients) 


Score  (Prompt  5)        Score  (Prompt  6,: 
(N  -  20)  (N  -  19) 


*  Significant  at  p^0.05 
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Legibility  0.24  0.29* 
Proportion  of  Total  Errors  to 

Total  Words  -0.14  -0.57* 
Proportion  of  Words  iu  Longest  3 

Chains  to  Total  Words  --0.18  -0*34  * 
Proportion  of  Words  in  Longest 

Chain  to  Total  Words  -0.15  «'0.37* 


Table  6 


Score  Related  to  Proir.pt  Based  Explicit,  Elaborated 
'    and  Inappropriate  Responses  by  Prompt 


(Kendall's  Tau^) 


Explicit 
(proportion) 


Elaborated 
(proportion) 


Inappropriate 
(proportion) 


Prompt  5 
(N  «  20) 

Pror[ipt  6 
(N  -  19) 


-0,47 


oai 


-0.50 


0,31* 


0.18 


-0.34* 


*  Significant  at 
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Table  7 


-Je::^t  Res-Its,  Means  and  Standard  Deviations  for  Explicit, 
Elab^fcsttrd  and  Inappropriate  Respc^nses  Across  Prompcs 


Ex 

pi icit 

Elaborated 

Inappropriate 

X 

SD  t 

X           SD  t 

X            SD  t 

Prorpt  5 

(H  =  20) 

0,01 

0.13 

0.01  C.Ol 

0.01  0.01 

-6.82* 

0.17 

4.23* 

Pror.f  r.  * 

(N  =  19) 

0.04 

'0.01 

0-01  0.01 

0.00  0.01 

*  Significant  at  p<^0.05 
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Chapter   3:      Part  II 

\  FTriT)  TE-^T  OF  EIGHT   PROMPTS:  SANTA 
■    CLARA  COUIJTV  WRITING  ASSESSMENT 

^  Gerald  Ca-p 

1 . :  ■    The  Scope  of  the  Study 

Tr.  th^  f^ll  of  IQ^Q    the  Bav  Area  Writing  Project  at  the 

.Titing  top.cs  or  promts     ^4^P-7l^3r^,,3  upon  which  . 
of  the  Consortium  s  bank  ot  ^vaiua  competency  testing, 

member  districts  ^^g^t  dravs  for  the  puipo  e  /    .  ^.^^h  the 

Specifically,  the  Bay  Area  Writu^g  Project  vsas  cnaig 
following  taslcs: 

1     Develop  seven  or  eight  questions  (prompts)  for  Avriting 
sSnple  to  be  field  tested  ;^th  ninth  and  tenth  grade 
students. 

-  Coordinate  field  testing,  with  assistance  of  county, 

-  using  s^-e  districts  involved  in  math  and  reading 
samples  (1977-78). 

3  Develop  a  resource  booklet  for  students  ^^^d  teachers 
outlining  procedures  for  scoring  each  prompt,  using 
SoHs?ic'g?ading  and  providing  sample  questions  to 

I  prepare  students  for  the  examination. 

4  conduct  holistic  grading  of  all  prompts,  .ith  standard 
point  scale  applied  to  each. 

thP  field  test.    Include  these  data 
^.    Tabulate  results  ot  the  tieiu  lcsl. 

in  booklet  (^3) . 
I„  .^sequent  discussions,  *-^Con=orti«  determined  tte^ 
.-as  net  fin-^jfl>.  r^^  i:,°ad  th    cS2crti™  decided  it  would 

ai^rin^^re'peSrrS^c^'r/s^ilenfs  In-'cther  matures  of  siting 

ability. 

For  the  purposes  of  selecting  the  lff^l,lf^''°J,\il\%r,g 
study,  BAUP  used  the  P":^-f^%^;^l°llZZlt  oi  Basic  Skills. 

i:l|,=rih;sf  ?^.:r^7^."percStile  were  designated  hi^- 
range. 

"  '  ^o,'oinnpd  bv  a  sub-conmittee  of  the  Santa 

The  prompts  >^'^^%developed  b)  ^^^  outstanding 

ErIc  lfT.l'^trT^il^lltXT.r^^  "f  the  Office  of  the  Santa 
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Clara  County  Superintendent  of  Schools-    (Coimiittee  members  who 
worked  on  the  development  of  the  pror^Dts  are  listed  in  Appendix 
A,)    The  BAiVP  consultant  furnished  the  cormittee  with  a  list  of 
pronpts  from  many  sources,  and  members  of  the  coinmittee  contrib- 
uted topics  they  had  used  in  their  classrooms  or  districts.  The 
BAWP  consultant  and  the  conmittee  met  several  times  during  the 
fall  and  winter  of  1979-80,  narrowing  the  list  of  proirpts, 
rewTitjng  each  of  those  finally  selected  several  times  until 
everyone  was  satisfied  that  each  proupt  was  expressed  as  clearly 
and  as  sinply  as  possible  and  would  present  few  problems  to 
students  to  whom  it  would  be  assigned.    Sources  for  "borrowed" 
prompts  are  listed  under  "References,"  page  379  below. 

Some  of  the  work  of  this  committee  can  be  seen  by  coirparing 
the  first  drafts  of  the  pronpts  to  the  final  drafts  actually  used 
in  the  field  testing.    Both  lists  follow: 


Santt  Qara  County 
Writing  Assessment  Prorapts 

First  Draft 

1.  There  are  probably  several  things  you  would  prefer  to  be 
doing  right  now  instead  of  sitting  here  writing  an  essay. 
Choose  ONE  of  these  things  and  describe  it  so  that  your 
reader  can  iinagine  it  as  well  as  you  do  and  can  under- 
stand why  you  yould  like  to  be  doing  it. 

The  activity  can  be  something  you  do  every  day,  or  some- 
thing unusual,  or  even  something  imaginary  that  you  may 
never  actually  do. 

2.  Imagine  that  you  and  three  other  people  are  trapped  in 
one  of  the  following  situations  and  you  cannot  escape 
for  several  hours.   Describe  your  feelings  or  tell  a 
story  about  what  you  do  to  escape  or  to  pass  the  time 
until  you  are  rescued. 

A.    an  elevator        b.   a  well      c.    a  locked  room 

d.   a  life*boat  e.  a  car   going  80  mph 

3.  If  you  had  to  choose  to  be  scmething  other  than  a  human 
being,  what  plant  or  animal  or  other  fom  would  you  choose? 
In  yxxjT  writing,  give  your  reader  some  idea  of  what  you 
think  it  would  be  like  to  be  that  fono,  and  of  why  you 
chose  it. 

4.  If  you  could  change  places  with  someone  else,  would 
it  be?  The  person  you  write  about  could  be  a  friend, 

a  member  of  your  fnily,  or  anyone  else,  living  or  dead 
or  imaginary,  drawn  fran  past  or  present,  fron  books, 
films,  etc.,  or  from  your  own  imagination.    In  your 
writing  give  your  reader  some  idea  of  what  it  would  be 
like  to  be  that  person,  and  of  why  that  life  appeals 
to  you. 
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5.  Think  of  a  personal  experience  that  has  in  sane  way 
changed  your  life,  either  for  better  or  for  worse: 

a  particular  event;  a  person,  a  place  you  have 
visited;  a  book  you  have  read;  a  film,  television 
show,  or  play  you  have  seen.   Describe  the  experience 
in  detail  and  explain  fully  why  it  was  an  important 
one  for  you. 

6.  People  scrcetiaes  do  things  that  make  others  angry, 
accidentally  or  intentionally.   Describe  a  situation 
that  has  caused  you  to  became  angry.   Explain,  using 
reasons  and  examples,  why  this  situation  angered  you, 
and  describe  how  you  dealt  with  the  problen. 

7.  While  visiting  in  your  grandmother's  house,  you  go  up- 
stairs into  the  attic.    In  a  dusty  comer  you  find  a 
large  trunk.   Propped  on  a  chair  oeside  the  trunk  is  a 
card,  and  on  it  in  your  grandmother's  handwriting  are 

the  words,  'To  my  only   grandchild:    Inside  this  trunk  you 
will  find  what  you  have  always  wanted.   Open  the  trunk 
and  remonber  me." 

Describe  the  gift  ani  how  you  will  use  it. 

8.  Think  of  a  friend,  real  or  imaginary,  that  you  had  when  you 
were  younger.    Describe  one  thing  you  and  this  friend  did 
together  or  might  have  done  together.   Try  to  show  your 
reader  the  kind  of  person  this  friend  was  and  why  you  chose 
this  person  as  a  friend. 


Letter  Prompts- 

9.   You  have  been  sent  a  bill  by  a  coraputer  charging  you  for 
two  record  albuns  you  did  not  purchase.   Write  a  letter 
explaining  the  problm  so  that  someone  in  the  billing 
department  can  correct  the  error. 

The  bill  is  from:  Melodie  Record  Club,  444  Tune  Street, 
Nashville,  Tcnn.  78QS6 

The  records  charged  to  your  account  are:    Kiee  Presents 
Opera  highlights  and  Led  Zepplin  Country  Bits. 

Your  name  and  address  are:  Chris  Brcwi,  456  Main  Street, 
Bay  City,  CA  92001 

Your  account  number  is:  235-9863422 
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10.    You  are  looking  for  an  interesting  way  to  spend  the  sumer. 
You  see  the  following  ad  and  decide  to  apply. 

Men- Women: 

TRAIN  NCW  FOR  THE  FUTJRE! ! 
APPRENTICES  WAKIH):    ^SUFflER  TRAI>aNG  PROGRAM  FOR  TF-ENS 

—Electrical  Engineer ing/Plu«bing/Carpentry 
—Recreation  Leadership/Wilderness  Guide 
—Veterinary  and  Zoo  Caretaking 
—Accounting  and  Computer  Pnogranming 
•-Motorcycle  and  Car  R^ir 
—Secretarial/Receptionist 
--Teacher /Public  Health  Assistant  Trainer 

June  20  through  August  20,  with  school -year  and  post- 
graduation  job  placement  guaranteed.  COST:  $500,  live-in. 

Send  resume  and  choice  of  program  to: 

TEEN  APPREOTICES 

P.O.  Box  1111 

Bay  City,  CA  97006 


Write  a  letter  applying  for  admission  to  this  sijmmer 
training  program.    Specify  which  part  of  the  program 
you  would  like. to  join,  and  what  interests,  eacperi-^ 
ences,  or  personal  characteristics  especially  qualify 
you.    Explain  whether  or  not  you  will  need  scholarship 
aid. 

Pretend  yocr  name  is  Chris  Browi  and  you  live  at  4  56 
Main  Street,  Bay  City,  CA  92001, 

11.  The  teachers  in  ytxir  school  have  asked  students  to 
submit  suggestions  about  how*  to  ijqprave  the  school. 
Think  of  ONE  and  only  one  problen  which  you  would 
like  to  see  solved  to  make  life  at  school  better  for 
you  and  others.    In  a  letter  to  the  faculty  describe 
the  problqn  you  would  like  to  see  changed,  give  any 
suggestions  you  might  have  about  how  to  bring  about 
the  change,  and  explain  how  your  suggestions  will 
affect  the  school.   Your  suggestions  may  be  serious 
or  humorous,  bwt  they  should  sound  sincere. 

12.  You  have  been  accused  by  a  neighbor  of  vandalizing  the 
neighbor's  car  with  a  group  of  friends.    Your  parents 
are  prepared  to  make  you  help  pay  for  the  damage.  You 
are  innocent.    Write  a  letter  to  the  neighbor  in 
which  you  persuade  him/her  of  your  innocence  and 
explain  what  you  think  should  be  done  about  the  matter. 
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FiMl  Draft  County 

Writing  Assessment  Pronpts 
(Revised  3/4/80  by  SCC  Writing  Committee) 

1.    Imagine .jthtt  you  are  trapped  in  one  of  the  following 
situations  and  you  cannot  escape  for  several  hours. 
Tell  what  you  do  to  escape  or  to  pass  the  time  until 
you  are  rescued. 

a.    an  elevator  b.    an  abandoned  mine 

c.    a  locked  room     d.    a  life  boat 


2.  If  you  had  to  choose  to  be  soroething  other  than  a  himan 
being,  what  %^5uld  you  choose?   Give  your  reader  some 
idea  of  irfiat  it  Moald  be  like  to  be  that  form,  and 

vrfiy  you  chose  it. 

3.  Think  of  a  personal  experience  that  has  in  sane  way 
changed  your  life:    a  particular  event;  a  person;  a 
place  you  have  visited;  a  book  you  have  read;  a  film, 
television  show,  or  play  you  have  seen.    Describe  the 
experience  and  explain  vAy  it  was  an  in^xDrtant  one  for  you. 

4.  While  visiting  in  your  grandmother's  house  you  find  a  large 
trunk.    Propped  on  a  chair  beside  the  trunk  is  a  card,  and 
on  it  in  your  grandmother's  handwriting  are  the  words,  "To 
my  only  grandchild:    Inside  this  trunk  you  will  find  what 
you  have  always  wanted.'' 

Describe  the  gift  and  how  you  will  use  it. 

5.  Think  of  a  friend,  real  or  imaginary,  that  you  had  v*ien  yew 
were  younger.    Describe  something  you  and  this  friend  did 
together.    Try  to  show  your  reader  the  kind  of  person  this 
friend  was  and  why  you  chose  this  person  as  a  friend. 

6.  The  school  newspaper  has  asked  students  to  sutmit  suggestions 
about  how  to  inprove  the  school.   Think  of  OtiE  and  only  one 
problem  which  you  would  like  to  see  solved  to  make  life  at 
school  better  for  you  and  others.    In  a  letter  to  the  school 
newspaper  describe  the  problem  and  tell  how  you  would  like 

to  see  it  solved. 

7.  Bnployment  agencies  can  be  very  helpful  in  finding  you  a  job 
if  they  know  what  job  you  want  and  are  qualified  for.  Write 
a  letter  to  The  ABC  Er^loyment  Agency,  204  East  Main  Street, 
Bay  City,  CA  94606.  Explain  what  job  you  would  like  and 
what  your  qualifications  are.  Explain  the  reasons  you  think 
you  should  be  hired  and  give  examples  of  your  strengths. 
Include  references. 

8.  You  have  been  sent  a  bill  charging  you  for  two  record  albums 
you  did  not  purchase.    Write  a  letter  explaining  the  problem 
so  that  someone  in  the  billing  department  can  correct  the 
error. 

The  bill  is  from  Melodie  Record  Club,  444  Tune  Street,  Nash-, 
ville,  Tenn.  78056. 

The  records  charged  to  your  account  are  Kiee  PreseKts  Opeva 
BighlightB  and  Led  Zepplin  Country  Hits. 

Your  name  and  address  are:  Oiris  Brown.  456  Main  fj!;"^' 
Bay  City,  CA  92001.    Your  account  number  is:  :)5-800-.l. 
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3. 0    Designing  the  Sairple 

To  ensure  an  adequate  number  of  student  papers  on  each  prompt 
to  provide  a  valid  sairple,  the  BAWP  consultant  worked  with 
Catharine  Keech,  BAWP  Research  Assistant  and  a  Ph.D.  candidate  in 
education  at  V.C.  Berkeley  specializing  in  writing  assessment. 
Her  analysis  of  the  necessary  numbers  for  adequate  sampling  is 
included  as  Appendix  B. 

Based  on  Keech 's  infonnation,  BAJVP  presented  the  Consortium 
with  three  sanpling  plans  from  which  to  select.  TTiese  plans  are 
all  based  on  a  field  test  of  the  original  twelve  pronpts;  subse- 
quently, the  Consortium  chose  to  test  eight  of  the  twelve. 

Plan  A:    Comprehensive  District  Sampling 

Will  provi^^^e  data  on  how  each  prompt  performs  in  each  district. 
Procedure 

1.  Sample  each  school  in  each  district  to  obtain  reliable 
district  data  on  each  profnpt. 

2.  Mininum  number  of  pap>ers:    12,000.    flininium  number  of 
classes  necessary  for  reading:    396  at  30  per  class. 

3.  Important  to  comninicate  with  a  responsible  person  at 
each  school  CEnglish  Department  Chairperson,  Director 
of  Guidance,  etc.)  to  insure  a  stratified  sample  (equal 
number  of  ninth  and  tenth  graders,  proportional  number 
of  high  achievers,  average  students,  lew  achievers). 

Plan  B:    Caunty^wids  Stratified  Sampling  by  Ability  Levels 

Will  provide  data  on  how  each  prompt  performs  county-wide  with 
students  identified  as  high  achievers,  average  achievers,  and 
low  achievers  and  with  schools  rated  higli,  average  and  low. 

Will  not  provide  data  on  how  each  prompt  performs  in  each  dis- 
trict. 

Procedure 

1.  Using  socio-economic  scale  and/or  school  averages  on  CTBS, 
categorize  each  school  as  high,  average  or  low  in  overall 
student  achiev€B«nt. 

2.  Detennine  the  proportion  of  schools  which  fall  in  each 
category.    Sanple  must  be  taken  frcmi  the  same  proportion 
of  schools  in  each  category. 

3.  For  the  smallest  category,  select  one  or  more  represen- 
tative schools.    Administer  the  writing  sample  in  classes 
identified  as  high  achievers,  average  achievers,  and  low 
achievers.    To  test  all  twelve  prompts  for  each  ability 
group,  we  will  need  twenty  papers  per  prompt  per  ability 
group,  or  240  papers  per  group  X  3  ■  720  papers. 
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For  the  mid  range  sample,  BA1\T  tested  eight  classes  divided 
among  three  schools,  again  selecting  one  school  in  the  34-44  per- 
centile range,  one  in  the  45-55  range,  and  one  in  the  56-66  range. 

For  the  high  range  sample,  50  percent  of  the  total,  BAW 
worked  with  teachers  in  fifteen  classes  divided  among  four 
schools:  one  in  the  67-77  percentile  range,  one  in  the  78-88 
range,  and  two  in  the  largest  category,  the  89-99  percentile 
range. 

As  closely  as  possible,  classes  were  equally  divided  between 
ninth  and  tenth  grade  classes  or  BAWP  used  classes  of  mixed  ninth 
and  tenth  grades.    Classes  tested  were  either  those  identified  as 
"average  ability"  classes  or,  in  schools  in  v*iich  conposition 
classes  were  tracked,  tests  were  given  in  one  ."low  ability' 
class,  two  "average"  classes,  and  one  "high"  class. 

Either  class  attendance  or  class  size  was  surprisingly  low 
on  the  days  the  writing  was  done;  as  a  result,  we  failed  to  get 
quite  as  many  samples  as  hoped  for.    In  the  smallest  cell,  the 
low  range  schools,  we  obtained  from  seventeen  to  twenty- six  papers 
on  each  topic;  in  mid  range  schools,  between  twenty- three  and 
thirty  papers  on  each  were  obtained;  in  high  range  schools, 
between  thirty-eight  and  forty-seven.    Despite  our  being  a  few 
papers  short  of  the  goal  on  a  few  of  the  topics,  the  range  of 
scores  is  such  that  the  results  are  adequate  for  field- test 
purposes. 

4.0   Conducting  the  Writing  Sanple 

In  each  district  in  which  the  writing  sanple  was  taken,  the 
Consortiim  coordinator.  Dr.  Burklund,  contacted  a  district  cur- 
riculum or  assessment  director  and  explained  the  project.  The 
BAWP  consultant  then  contacted  the  district  officer,  the  principal 
at  each  designated  school,  and  the  English  department  chair  person, 
who  selected  the  classes  to  be  sampled.    The  teacher  of  each  class 
was  sent  a  collated  packet  containing  an  equal  nunber  of  test 
papers  on  each  topic.    Each  teacher  also  received  the  following 
instructions: 

It  i»  ijnportjmt,  to  provide  valid  results,  that  the 
sampling  be  done  as  uniformly  as  possible.  Here  is 
vthAt  I  would  like  you  to  do: 

1.  Select  a  day  during  the  week  of  April  21-25  when 
you  can  devote  the  whole  period  to  the  writing 
sample. 

2.  Oh  the  day  before,  tell  your  students  what  they 
will  be  doing  on  the  following  day.    Tell  them 
the  following: 

a.    They  will  be  participating  in  an  (Experiment 
by  the  Universrity  of  California  to  see  what 
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4.   Administer  writing  sample  in  the  other  two  school  categories 
ind  ability  groups  in  proportion  to  the  county-wide  distri- 
bution of  schools  in  each  category.    (Thus  if  the  smallest 
groxjp  of  schools  is  those  identified  as  low  achievers,  we 
wuld  have  72Q  papers  representing  that  group,   If  there 
are  tidce  as  many  "average  achiever"  schools,  we  would 
need  144Q  papers  from  those  schools,  and  so  onO  The 
total  mjuber  of  papers  in  the  sample  will  fall  between 
216Q  (if  each  group  is  roughly  the  same  size)  to 
probably  no  more  than  5000. 

Plan  C:    Connttf-vide  Stratified  Sampling 

Will  provide  data  on  how  each  pronpt  perfoms  county-wide  in 
schools  identified  as  high,  average,  or  low  in  overall  student 
perfoTinance. 

Will  not  provide  data  on  students  identified  as  high  achievers, 
average,  or  low  achievers. 

Procedure 

1.  Categorize  schools  and  determine  proportions  as  in  number 
one  and  number  two  above. 

2.  For  the  smallest  category,  select  one  or  more  representative 
schools  and  eight  or  nine  representative  classes.  Administer 
writing  sample  to  obtain  twenty  papers  per  pronpt,  or  a 
total  of  240  papers. 

3.  Administer  writing  san?3le  in  the  other  two  school  categories 
in  proportion  to  the  county-wide  distribution  of  schools  in 
each  category.    The  total  number  of  papers  in  the  sample  will 
fall  between  720  Cif  wch  group  is  roughly  the  same  size)  to 
probably  no  more  than  2000. 

The  Consortivjn  deteiroined  that  Plan  C  woiild  provide  adequate 
data  and  that  eight  proirpts  should  be  tested. 

Following  this  decision,  BAWP  determined  the  configuration 
of  the  sample  by  u^ing  the  1977-78  data  on  the  performance  of 
seniors  in  county  schools  on  the  writing  skills  portion  of  the 
California  Assessment  of  Basic  Skills.    Using  these  figures  RAW 
calculated  that  of  the  forty- four  secondary  schools  from  which 
data  was  available,  ten  schools  or  23  percent  fall  within  the  low 
range  (1-33  percentile),  twelve  schools  or  27  percent  fall  within 
the  mid  range  (34-66  percentile) ,  and  twenty-two  schools  or  50 
percent  fall  within  the  high  range  (67-99  percentile). 

Based  on  the  need,  then,  for  twenty  papers  per  topic  from 
the  smallest  cell  (low  range  schools)  or  a  minimum  of  160  papers, 
BAIN-P  determined  to  sample  seven  classes  divided  among  J^ree 
schools.    To  help  ensure  a  stratified,  one  school  in  the  1-11_ 
percentile  range,  one  in  the  12-22  range,  and  one  in  the  o-j^"^ 
range  were  chosen. 
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Icinds  of  topics  students  like  to  write  on 
and  which  kinds  produce  their  best  writing. 

b.   Their  names  will  not  be  on  the  papers,  so 
the  teachers  reading  the  papers  will  not 
know  who  they  are.    (Readers  won't  even 
know  what  school  the  papers  are  from.) 

•c.   They  will  put  numbers  on  their  papers  so  the 
teacher  can  return  papers  tc  students.  CYou 
will  receive  the  unmarked  originals  to  use  in 
whatever  way  you  see  as  valuable.) 

d.  Different  students  in  the  class  will  be 
writing  on  different  topics. 

e.  They  will  be  given  thirty  minutes  to  write 
the  papers. 

f .  Each  student  should  be  sure  to  bring  a  pen 
or  a  dark  (number  2)  pencil  to  write  with. 
(Writing  mist  be  dark  enough  to  photocopy 
or  we  won't  be  able  to  return  the  original.) 

3.    On  the  day  you  take  the  san^jle,  follow  these  steps; 

a.  Be  sure  everyone  has  a  pen  or  a  number  2  pencil 

b.  Pass  out  the  -test  fonns.    (They  are  collated 
so  that  we'll  get  an  equal  number  of  each 
topic.    Try  to  pass  them  out  in  the  order 
they're  in.)  ^ 

c.  Tell  the  students  to  fill  in  the  following 
numbers  on  the  top  page: 

School  Code    Teacher   Class  


For  Student  IhorttTt  give  each  student  his/her 
number  in  your  roll  book. 

d.  Tell  students  also  to  write  their  Student 
fhanber  and  Clast  nunber  on  the  top  of  page  2. 

e.  Tell  students  to  raise  hands  if  they  have  a 
question  about  their  topic.    Deal  with  those 
individually.    (Involving  the  whole  class  in 
disciission  of  a  topic  only  a  few  have  would 
be  confusine.') 


•It  later  proved  unfeasible  to  return  the  originals. 
Several  factors  played  a  part  in  the  decision  to  retain  them: 
1)  the  tijT»e  involved  in  photocopying  all  papers,  2)  the  fact  that 
sane  could  not  be  photocopied,  3]  considerations  of  topic  security. 
In  lieu  of  their  originals,  participating  teachers  were  sent  the 
list  of  practice  topics  and  the  scoring  criteria  so  they  might 
practice  the  holistic  scoring  procedure  idth  their  classes  if  they 
chose. 
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f .  Have  a  supply  of  extra  paper  available  in  case 
anyone  needs  it.    Tell  them  to  raise  hands  if 
they  need  an  extra  sheet.    Try  to  see  that 
students  adding  a  sheet  put  their  student  number 
on  the  sheet. 

g.  Tell  them  they  have  thirty  minutes  and  to  begin 
writing. 

h.  At  the  end  of  thirty  minutes,  call  time.  Allow 
students  to  finish  the  sentence  they  are  writing, 
then  collect  the  papers. 

i.  If  tijne  renains,  feel  free  to  permit  students 
to  discuss  the  experience,  compare  notes  on 
topics,  etc. 

5.0   Selecting  Anchor  Papers 

When  all  test  papers  had  been  returned  to  the  county  office, 
the  BAWP  consultant  met  with  two  members  of  the  County  Writing 
Conmttee,  Rosalie  Chako  of  Saratoga  Elementary  District,  and 
BAWP  teacher/consultant  Irvin  Peckham  of  Morgan  Hill  Unified,  to 
select  anchor  papers  for  the  scoring. 

Personal  essay  papers,  topics  1-6,  were  first  separated  from 
business  letter  papers,  topics  7-8.    Each  of  the  three  readers 
quickly  read  through  roughly  forty  papers  each,  dividing  the 
papers  into  two  groiq^s:  top-half  papers  and  bottom-half  papers. 
Scoring  was  then  refined,  each  paper  finally  receiving  a  rank 
from  one  (lowest)  to  nine  (highest) .    The  readers  exchanged 
papers  and,  without  seeing  the  previous  ranking,  reranked  the 
new  group.    They  then  exchanged  a  third  time,  so  that  all  papers 
had  been  read  and  ranked  by  all  three  readers. 

Papers  which  all  three  readers  had  ranked  the  same  were 
considered  anchor  papers.    Those  all  three  had  ranked  2,  4,  6, 
and  8  were  analyzed  and  their  identifying  qualities  were  listed. 
These  lists  of  qualities  became  the  scoring  guidelines  (rubrics) 
for  readers. 

A  sajTpling  of  business  letter  papers  was  then  read  by  the 
three  readers,  anchor  papers  were  selected,  and  scoring  guide- 
lines written. 

Before  the  day  of  the  reading,  BAWP  made  photocopies  of  the 
selected  anchor  papers  to  use  to  train  the  readers. 

Below  are  the  scoring  guidelines  for  the  reading  in  general 
and  for  the  two  types  of  topics.    Anchor  papers  for  each  prornpt 
will  be  found  following  the  data  on  the  prompt. 
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Santa  Qara  Coimty  Writing  Assessnent 
Scoring  Guidelines 

1.  Use  scale  of  1-9,  but  use  even  points  primarily.  First 
decision:    is  paper  in  upper  half  or  lever  half?  Is 
paper  a  2,  4,  6,  or  8?   Ihen  refine  scoring,  using  odd 
nunfcers  to  reward  things  which  really  stand  cut —language 
use,  freshness,  crispness,  or  to  lorer  the  score  of  a 
paper  which  was  a  struggle  to  read. 

2.  Reward  the  writer  for  what  he/she  does  well.    Score  all 
papers  which  can  be  seen  in  any  \vay  as  responses  to  the 
topic.    A  writer  may  argue  with  the  topic  and  still  re- 
ceive a  top  score.    Read  quickly;  do  not  inark  the  papers 
in  any  way.    Do  not  attempt  to  tally  errors;  give  each 
paper  a  rapid-is^rression  score,  then  go  on  to  the  next 
paper. 

3.  Generally  ignore  spelling  and  punctuation.   Add  a  point 
for  exceptionally  sophisticated  style  and  grace;  subtract 
a  point  for  severe  mechanical,  graimiatical,  and  stylistic 
deficiencies  that  block  cojimmication  of  meaning. 

4,  Do  not  be  prejudiced  by  handwriting^  unless  the  paper  is 
completely  illegible. 


Essay  Scoring 

VhsLX  follows  are  general  descriptions  of  the  kinds  of  papers  yoa 
are  likely  to  find  in  each  scoring  range.    Do  not  feel  con- 
rtrained  by  these  guidelines,  however,  if  you  find  papers  whica 
illustrate  strengths  or  weakness«s  not  mentioned. 

8  -  Topic  is  fully  developed  with  a  beginning  and  a  sense  of 
ccnpletion.    Paper  leaves  reader  with  an  emotion  if  a 
narrative  paper,  or  persuades  reader  if  a  persuasive  paper. 
Writer  uses  precise  diction  and  is  imaginative  in  choice  of 
words,    Paper  has  a  sense  of  sincerity  and  will  exhibit 
vivid  descriptions  or  careful  reasoning. 

6  -  Paper  sticks  to  topic  and  develops  it  to  some  degree  with 
specific  details  or  examples  and  with  sane  feeling  evident. 
Paper  has  a  generally  adequate  opening  and  closing.  Sen- 
tences are  varied  and  generally  correct,  though  there  may 
^e  some  awkward  sentences.  Diction  is  accurate  but  shows 
i-ittle  flair  for  words. 

4  ---Sonewhat  thin  in  content  but  uses  some  sDCcific  details. 
Little  sentence  variety.    Sentences  are  generally  complete, 
thou>?h  there  are  likely  to  be  awkward  sentences  and  cc»iima 
splices.    Papers  will  often  have  a  weak  or  unimaginative 
opening  or  closing  and  may  exhibit  diction  problems  or 
grammatical  errors. 
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2  -  VcTy  thin  content,  not  many  concrete  details.  Papers  will 
be  on  the  topic  and  will  develop  it  in  at  least  one  para- 
graph of  five  or  loore  sentences.  Sentence  structure  will 
often  be  iinnature.  Papers  T»y  have  very  minimal  openings 
and  closings.  Often  writers  will  omit  words  or  word  end- 
ings- This  score  should  be  used  for  papers  that  reveal 
serious  second^language  probleois. 

0  -  Blank  papers  or  papers  which  completely  ignore  the  topic. 
CNote:    Take  any  0  papers  to  the  head  reader.) 

Busineaa  Letter  Scoring 

In  general,  mechanical  correctness  will  weight  mere  heavily  in 
scoring  essays.    Also  to  be  rewarded  are  correct  business  letter 
foTTTi,  concise,  to-the-point  clarity,  and  meeting  all  of  the 
requiresBcnts  of  the  topic. 

8  -  niCistrates  correct  and  ccinplete  business  letter  form 
including  return  address,  inside  address,  appropriate 
ODening  and  closing.    Complaint  letter  will  contain 
ail  information  required  in  the  topic,  and  will  organise 
the  information,  clearly,  concisely,  and  correctly. 
i^^Tplication  letter  may  show  soroe  imagination,  but 
iflust  include  all  required  itms,  and  must  not  ramble 
on  aimlessly - 

6  -  Shows  some  biowledge  of  correct  business  letter  form, 
though  parts  of  form  not  essential  to  delivery  and 
accomplishment  of  goal  may  be  oniitted.    Do  not  score 
any  paper  in  the  i^per  half  which  would  not  accomplish 
the  assigned  task.    Papers  receiving  this  score  will, 
in  general,  not  be  as  polished  afid  clear  as  S  papers. 

4  -  Business  letter  form  not  followed,  or  followed  very 
inperfectlv.    Most  information  required  by  the  topic 
is  included,  but  may  be  poorly  organized  or  not  clearly 
expressed. 

2  •  Very  inadequate  as  a  business  corarmnication;  important 
itCTS  of  information  omitted;  generally  very  poorly 
organized,  with  little  or  no  sense  of  letter  form. 

Q  -  Blank  or  non-responsive  papers.    CTake  to  head 
reader.) 


6.n    Scoring  the  Papers 

Kate  Sutherland  invited  teachers  from  several  county  school 
districts  to  participate  as  readers  in  the  holistic  scoring  oi 
the  papers.    Most  but  not  all  of  the  readers  were  members  of  the 
Santa  Clara  County  Writing  Coimiittee.     CNames,  schools,  and 
districts  of  readers  are  listed  in  Appendix  C.) 
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Readers  met  at  the  Office  of  the  Santa  Clara  Countv  Superin, 
tendent  of  Schools  at  9:00  A.M.  on  Saturday,  May  10.  Readers 
were  given  copies  of  the  scoring  guide  for  essay  scoring  and 
anchor  papers  representing  scores  of  2,  4,  6,  and  8.    Each  reader 
ranked  the  anchor  papers  according  to  the  criteria  on  the  scoring 
guide.    The  head  reader  (Gerald  Cainp)  and  the  tv/o  table  leaders 
{Rosalie  Chako  and  Irvin  Peckhojn)  led  the  discussion  of  the 
anchors.    Readers  were  then  given  another  sairple  paper  to  score. 
VvTien  every  reader  had  assigned  the  paper  a  score,  all  scores  we  re- 
tallied  on  the  chalkboard.    Readers  were  called  upon  to  justif)' 
their  scores,  and  the  ensuing  discussion  served  to  bring  the 
group  closerffo  consensus. 

Six  to  eight  additional  samples  were  scored  and  discussed  in 
the  same  way.  When  the  head  reader  was  confident  that  all  reader, 
understood  and  werit consistently  applying  the  scoring  criteria,  h( 
distributed  the  papers  for  scoring. 

During  the  scoring,  the  two  table  leaders  frequently  rescore 
randoinly  selected  papers  fran  readers  at  their  tables.    If  a 
reader  seemed  to  be  consistently  scoring  too  high  or  too  law,  ^ 
the  table  leader  or  head  reader  would  review  the  scoring  criteria 
with 'him  or  her. 

After  reading  for  an  hour,  readers  took  a  brief  coffee  break 
Following  the  break,  the  entire  group  again  scored  and  discussed 
several  sanple  papers  to  help  keep  everyone  on- target. 

All  papers  received  at  least  tv/o  readings.    A  bundle  of 
twenty- five  papers  which  had  been  read  at  one  table  would  be 
transferred  to  the  other  table  for  a  second  reading.    Each  table 
had  a  code  to  use  in  scoring  so  the  second  reader  would  not  be 
influenced  by  the  score  assigned  by  tl^  first  reader. 

Papers  that  received  scores  more  than  one  point  apart  were 
returned  to  tJie  table  leader  for  a  third  reading.    If,  after  the 
readings,  two  scores  were  identical,  the  paper  was  assigned  that 
score.    If  the  paper  had  three  scores,  the  middle  score  was 
assigned. 

Readers  finished  all  personal  essay  papers  (topics  1-61, 
giving  each  two  or  three  readings  as  necessary,  shortly  before 
noon.    Following  a  lunch  break,  the  readers  reconvened,  were 
retrained  to  score  business  letter  papers  vising  the  business 
letter  scoring  guide  and  anchor  papers,  and  finished  the  entire 
reading  at  about  3:00  P.M. 

At  the  completion  of  the  scorijig,  readers  were  given  a 
questionnaire.    Questions  and  a  sunrnan'  of  responses  were  as 
follows: 

r 

1.    Did  vou  feel  that  anv  of  the  topics  produced  noticeablv 
weaker  writing  than  others?    If  so,  which  ones'?  Why 
you  think  this  was  so? 


_    h:-l  v:>u  ievl  that  anv  of"  rh-  toi'ic^  prrj:r.t.-j  jv.^  i     ^  i  - 
stronger  writmp  than  ethers''    If        wh.:  :h  -m----^  IVi..- 
do  you  thirik  this  was  so? 

There  was  very  little  agreement  ajnong  readers  as  to  uri.-h 
of  the  proni^ns  were  less,  which  more  successfuP.    Of  the  pron^  t:- 
iTientionedj  only  number  two,  "If  you  had  to  choose  to  be  soinething 
other  than  a  human  being/'  received  a  significantly  negative  rear 
tion.    Seven  readers  felt  it  was  a  poor  topic  whereas  only  one 
reader  listed  it  as  among  the  best-    Reasons  given  for  the  nega 
tive  reactions  generally  suggested  the  topic  was  not  suitable  for 
high  school  students,  ''a  worn  out  topic  from  elementar)'  days''  or 
that  many  student  \^Titers  failed  to  give  reasons  for  their  choice 
of  particular  forriS. 

On  the  positive  side,  six  readers  felt  that  topic  number  six 
':V*  letter  to  the  scJiool  newspaper  suggesting  a  change,  produced 
'  *rong  wTiringg  hut  two  other  readers  identified  it  as  a  weaker 

i>^^nce.ming  the  other  tojncs  there  was  even  less  consensus. 
r-.r^:ij  one,  ''Imagine  you  are  trapped.../'  was  identified  as  a  we^il 
topic  by  two  readers,  a  strong  topic  by  five.    Topic  three,  a 
personal  experrence  that  changed  your  life,  received  no  negative 
notices  and  three  positive.    Topic  four,  grandmother's  tnznk,  was 
identified  as  a  strong  topic  by  three  readers  and  as  a  weak  topic 
hv  one.    Topic  five,  a  childhood  friend,  was  seen  as  weak  by  four 
and  strong  by  five  responders,  topic  seven,  the  letter  of  applica 
tion,  received  one  negative  response,  and  topic  eight,  the  com 
plaint  letter,  received  one  positive  response.    A  general  comment 
on  business  letter  pron^ts  deserves  to  be  quoted,  as  it  reflect? 
a:i  opinion  mentioned  orally  by  several  readers  including  the 
table  leaders 5  an  opinion  strongly  held  by  many  teachers  of 
i^rriting: 

I  fomd  the  business  fom  letters  to  be  counterproductive. 
r<ather  than  encouraging  written  expression,  they  seemed 
to  stifle  SOTie  students  and,  by  the  nature  of  the  proiipt, 
penalize  any  attenpt  to  editorialize,  criticize,  or  exp»and 
the  writing  in  any  way.    I  found  the  skill  to  be  too 
^viractured  to  be  a  measured  writing  skill. 

From  the  mixed  reactions  of  the  readers  we  can  conclude  that 
till  topics  with  the  possible  exception  of  topic  two  were  success- 
ful m  producing  the  whole  range  of  writing  quality  desired  in 
such  a  sampling.    It  is  interesting  to  note  that  the  statistical 
data  (see  below,  pp.  35^  374)  ^^Id  indicate  that  the  reaction  of 
the  readers  is  not  borne  out  by  the  scores  they  assTgned.  Scorei 
on  topic  two  follow  a  distribution  pattern  much  like  those  of  the 
other  topics.    One  might  speculate  that  many  ninth  graders  are  ir 
fact  more  irmature.  than  the  readers  would  like  them  to  be  and  car 
m  fact  respond  well  to  a  topic  some  readers  judge  to  be  better 
suited  to  younger  .'Children. 
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Are  there  any  topics  you  feel  should  be  re-worded  for 
clarity  or  to  make  them  easier  for  students  to  under- 
stand?   Can  you  suggest  re-wording? 

Few  responded,  and  the  suggestions  generally  were  vague. 
The  most  useful  suggestions  are  that  topic  seven,  the  emplov-ment 
agency  letter  pronpt    should  include  an  imaginary  name  and  return 
address  similar  to  that  in  topic  eight.    I  would  also  suggest 
adding  the  words,  "Use  correct  business  letter  form"  to  both 
seven  and  eight. 

4.    What  is  your  general  feeling  about  the  quality  of 
vrriting  by  the  students? 

Four  readers  felt  the  student  writing  was  "average,"  "ade- 
quate," "reasonable,"  and  "about  \Aat  I  expected."   Eight  people 
were  very  positive  about  the  papers  they  read,  indicating  that, 
given  only  thirty  minutes,  most  students  had  responded  surpris- 
ingly well.    There  were  only  two  negative  responses:  "Students 
show  poor  ability  to  organize  thoughts.    They  also  don't  develop 
ideas  well."    "On  the  whole  the  corps  revealed  a  poverty  of  mind. 
As  head  reader,  having  participated  in  many  readings,  I  was  per- 
sonally very  impressed  w.th  the  excellent  quality  of  the  iriajority 
of  student  papers. 

How  did  vou  feel  about  the  method  of  conducting  the 
reading?'  If  it  were  to  be  done  again,  v^iat  suggestions 
would  you  make  for  improving  the  procedures? 

Four  people  suggested  ijnprovements:  (1)  eliminate  codes  for 
socres  and  use  a  more  accurate  way  of  identifying  who  read  par- 
ticular papers  (from  a  table  leader);  (2)  have  all  third  readings 
of  papers  done  by  table  leaders;  (3)  eliininate  anchor  papers;  and, 
(4)  better  rooiri,  better  coffee.    The  other  ten  responses  were 
entirely  positive,  several  readers  noting  that  they  had  learned 
techniques  which  would  be  useful  to  them  in  working  with  students 
or  with  other  teachers. 


"Jl  Results  of  the  Study 

Results  of  the  scoring  of  each  of  the  prompts  is  given  here. 
Results  are  given  in  both  the  number  of  papers  receiving  «ach 
score  and  the  percentage  of  the  total.    Papers  which  received  two 
^core^  one  number  apart  are  rounded  downward  for  lower-half  papers 
and  upward  for  upper-half  papers.    Papers  scored .three  times.Wiich 
received  three  different  scores  were  assigned  the  middle  scote.  _ 
Scores  of  0  indicate  blank  papers  or  papers  that  were  completel> 
non- responsive  to  the  prompt. 

Following  the  statistical  report  on  each  prompt  are  several 
papers  representing  scores  of  2,  4,  6,  and  8  for  that  prompt. 
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Topic  2,    Ijnagine.  that  ypu  are  trapped  in  one  of  the  following 
.  situations  and- yiixi.  cannot  escape  for  several  hours. 

Tell  ^lAat  you  do  to  escape  or  to  pass  the  time  until 
■  you  are  rescued. 


t.  an  elevator 

'     "     •   c«  a  locked  room 

Low- Range  Schls*  Mid-Rarige  Schls- 
(1-33  percentile)  (34-36 -percent  •) 

Score'     No..  Percent  ,   No,  Percent 

^0         0  0- 

10'  0 

2  Z'  11.  2  7 

3  S  '26  7  24 
4.5  2&            7  24  ' 

5  2  11  2  7 

6  3  16  5  n 

7  2  11  3  10 
8^0  27 
9          0  13 


TOT.AL  19 

Sanple  Papers 
Low  -  Score  2 


29 


b.    an  abandoned  mine 
'd.    a  life-boat  ^ 

High- Range  Schls 
C67-99  percent  J 
No,  Percent 

0 

1  2. 
4  10 
6  15 

11  '  27 

9  22 

6  15  ' 

2  5 
1  2 
1  2 

41 


T  m  an  elevator,  trapped  and  I  can^t  get  out,  It  has 
been  along  time  since  I  been  in  here  and  nobody  has 
cj9ne  to  my  rescue.    But  I  can't  wait  no  inore  I  im 
going  to  try  and  get  out,  but  I  can  not  find  no  way 
out;  and  I  am  scard   I  don't  no  what  do.    May  be  I 
should  wait  intill  someone  comes  to  Tny  recue.  But 
I  been  here  a  long  tune  and  nobody  has  came,  so  then 
I  find  a  way  out,  it  is  on  the  of  the  elevator  there 
is  like  a  little  door  there,  I  am  going  to  try  and 
get  up  there,  but  it  is  hard  to  do.    Because  it  is 
to   high  and  I  cannot  reach  It,  but  I  try  and  get 
up  by  th05e  hand  holders,  then  I -reach  the  top  and 
try  to  open  the  door  but  Its  to  hard   but  I'm  going 
to  keep  on  trying  till  I  do  it     Then  finally  It 
open,  and  Im  trying  to  get  up  there,  Then  when  I 
get  up  there  the  door  open  and.  the  men  where  stand- 
ing there  who  fixed  it  and  told  me  to  get  down  fron 
there  and  the  would  help  me.    Then  when  I  got  there 
were  people  there  looking  at  me  and  asking  me 
qiiesten,  like  if  I  was  scard  and  how  do  I  fill,  and 
I  told  them  hour  I  felt  and  I  felt  scard  arxi  sick, 
that  I  need  to  lay  down  so  they  took  me  to  a  room 
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so  I  ccaild  lay  dwn  for  a  ccupie  of  minutes  so  I 
could  get  better. 

Lower  Half  -  Score  4 

We  were  trapped  in  a  locked  room  for  four  hours. 
At  first  we  thought  that  we  would  die  but  then  Phillip 
suggested  that  we  try  to  find  sonething  to  do.    We  all 
started  to  look  for  anything  we  could  do,  then  Nanc>' 
found  a  deck  of  cards.    After  playing  for  about  one 
arvd  a  half  hours  everyone  was  bord. 

Later  we  were  getting  pretty  scared,  we  thought 
we  would  never  get  out.    Then  Phillip  said  we  cann't 
just  stand  around  staring  at  each  other  because  it 
would  only  make  it  worse.    We  started  telling  each 
other  about  what  we  were  going  to  do  as  soon  as  we 
get  out.    All  of  us  had  some  plans  and  we  all  started 
to  cheer  up.    Us  girls  would  have  really  had  paniced 
if  our  boyfriends  hadn't  had  been  there,  but  they  inade 
us  feel  safe  with  them.   They  suggested  that  us  girls 
try  to  sleep  for  awhile  while  they  figure  out  what  to 
do.    That  was  the  last  I  remember  when  us  girls  vroke 
\jp  we  were  in  the  other  roan  and  the  guys  were  eating 
pizza.    Up  until  this  day  they  have  never  told  us  how 
they  got  us  out  but  I  am  sure  glad  they  didc 

Upper  Half  •  Score  6 

If  I  was  trapped  on  a  life  boat,  I  \^uld  try  to 
paddle  my  way  to  an  I-Sland  or  the  mainland.  Hor- 
ever  if  I  did  not  have  a  paddle  things  would  look 
bleak  for  awhile.    I  would  let  the  waves  carry  me 
somewhere.    But  I  am  not  sure  I  would  do  that.  In 
order  to  soothe  myself  down  I  would  listen  to  the 
radio  and  see  if  any  planes  would  be  flying  over  by 
where  I  am,    I  would  smy  to  Tnyself  while  I  am 
listening  to  the  radio  I  might  as  well  prepare  ray- 
self  for  the  liorst.    Put  on  a  life  jacket,  keep  a 
cool  head,  keen  the  boat  as  well  balanced  as  possible 
to  keep  niost  of  the  vater  out.   After  awhile  I  mght 
doze  off.    But  I  shouldnt  let  that  happen.  Some- 
thing could  happen,    I  might  float  out  to  sea.  Or 
worse.    In  order  to  keep  myself  awake  I  would  do, 
what  else,  but  splash  water  on  my  face.    If  a  plane 
did  not  come  for  awhile  I  would  float  around  until 
I  find  something,  like  a  piece  of  wood  for  example, 
to  paddle.    I -everything  else  fails,  I  would  probably 
have  to  use  my  arms  as  a  paddle    I  could  do  50  possibly 
and  rest  and  do  another  50  until  I  reach  San  Francisco 
Bay  and  get  help.    They  probably  woild  think  it  is 
fantastic  that  I  had  survived. 


High  -  Score  8 


It  vas  a  cKilly  night  Pt  the  sky  was  clear.  The 
water  belc^  seemed  like  a  cold,  depthle^s  pool  of 
death.    It  came  so  i^uick  S  so  unexpected,  a  giant 
boom  from  nc^here.    I  remmber  hitting  the  freezing 
water  wondering  lAat  was  going  on.   After  a  few 
swonds  of  shock  I  saw  a  yellow  bobing  object.  I 
was  so  cold  I  found  it  hard  to  move  about.   When  the 
yellow  object  approached  ine,  I  discovered  it  was  a 
lifeboat.    I  climbed  in  and  looked  around  for 
other  people  but  all  I  saw  was  burning  wreckage 
all  around  me.    After  a  yAiile  all  the  flamts  died 
out.    It  was  only  me  5  my  lifeboat,    ix  soon  became 
day  S  I  was  cold.    I  was  almost  blue.  Eveiything 
around  ©e  was-  ^ite  except  for  the  gray  nurky 
fathoms  of  never  ending  water  below  me.    Soon  bordom 
overtook  me.    I  began  counting  the  pieces  of  boards 
and  other  miscelanious  objects  that  would  pass  by 
every  once  and  a  while,    I  tried  singing  but  that 
only  made  me  feel  lonelier.    I  drifted  on  for  days. 
I  found  it  almost  impossible  to  sleep  when  I  thought 
of  the  cold.    After  awhile  though  I  forgot.    I  went 
to  sleep  and  felt  iny  body  try  to  shiver  off  the 
cold.    I  couldn't  control  these  strange  toovements  I 
was  making.    Finally  I  stopped  trying  5  didn't  care 
anymore.    I  went  to  sleep  and  I  still  am. 

Topic  2.    If  you  had  to  choose  to  be  something  other  than  a 
human  being,  what  would  you  choose?   Give  your 
reader  some  idea  of  what  it  would  be  like  to  be 
that  form,  and  why  you  chose  it, 

Low-Range  Schls.     *^d- Range  Schls.    High-Range  Schls. 


Score 

No. 

Percent 

No. 

Percent 

No. 

Percent 

0 

0 

Q 

1 

3 

1 

0 

0 

1 

3 

2 

2 

12 

0 

1 

3 

Z 
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12 

4 

IS 

8 

21 

4 

6 

35 

7 

26 

3 

8 

S 

1 

6 

7 

26 

8 

21 

6 

3 

IS 

5 

19 

11 

29 

7 

1 

6 

3 

11 

5 

13 

8 

2 

12 

1 

4 

0 

9 

0 

0 

0 

TOTAL 

17 

27 

38 
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Sample  Papers 
Low    -  Score  2 


If  I  had  a  choose  to  be  something  other  than  a 
human  being  I  would  what  to  be  a  little  rabbit.  On 
Easter  the  people  like  to  play  with  the  rabbit.  There 
so  pretty  and  fluffy  looking.    The>'  grow  up  to  be  very 
big  aivl  very  pretty.    They  like  to  play  around  with 
people  and  with  kids.    That  is  why  I  vrould  like  to  be 
a  little  rabbit. 

I  choose  it  because  it  is  a  beautiful  animal. 
They  like  to  play  around.    There  not  to  missy  and 
hardly  have  to  clean  up  after  the  rabbit.    They  eat 
all  their  food  you  give  them.    Thats  they  I  choose 
to  be  a  rabbit. 


Lower  Half  -  Score  4 

If  I  wasn't  a  human  I'd  be  a  quarter  hcurse 
because  I  would  like  to  go  to  the  Kentucky  derby 
sane  day.    I  would  run  as  fast  as  a  could  to  win  a 
race.    I  would  never  buck  or  kick  any  body  off.  Then 
maybe  I  would  make  it  to  the  Kentucky  derby. 

Once  I  make  it  to  the  Kentucky  derby  it  would 
be  tard.    I  would  be  trained  a  couple  more  mouths 
antil  I  beat  my  on  record  time, 
k  It's  the  day  of  the  race  the  grand  stands  are 

'  full  just  to  see  the  race.   Then  I  would  walk  onto 

the  track  to  let  the  people  in  the  grand  starts  see 
me.    Then  I  would  go  to  the  starting  gates  and  the 
race  would  begii..    I  would  try  as  hard  as  I  could 
to  win. 

Upper  Half  -  Score  6 

If  I  was  something  other  than  a  hman  being,  I 
think  I  would  like  to  be  a  bird.  If  I  were  a  bird  I 
could  fly  high  above  the  crowded  cities,  never  stop- 
ping until  I  was  far  past  them  into  the  wilderness. 
It  would  make  me  feel  so  fre*  and  no  one  could  make 
me  land  until  I  wanted  to.  I  really  appreciate  the 
freedom  birds  have  and  could  sit  and  watch  the*  all 
day. 

As  a  bird,  I  could  fly  high  above  the  world  -and 
look  at  everything  from  a  veiw  that  not  many  people 
inve  the  chance  to  see  it  from.    If  I  flew  high 
enough  I  could  see  great  mountains  as  tiny  hills. 
xni  see  giant  lakes  as  only  droplets  of  water. 

The  feeling  of  freedan  and  indepenance  would 
be  overwhelming.    I  could  souer  high  above  the  clouds 
for  almost  an  etumity  of  time,  landing  only  to  drink 
aid  eat.    I  vould  be  my  own  master,  flying  only  with 
I  my  thoughts  as  company. 

^  Birds  are  one  of  the  most  beautiful  and  grace- 

ful animals  on  earth,  swooping  and  diving  at  will,  in 
t}«  sky.    To  watch  a  bird  makes  roe  feel  as  free  in 
^  thouEht  as  they  are  in  flight.  ')  7  j 
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So  a  bird  I  wuld  experience  many  beautiful 
things  and  have  the  freedom  to  choose  between  them. 

High  -  Score  8 

Mother  Nature,    Yes^  that's  what  Td  choose 
to  be,    I'd  fill  the  prairries  with  stalks  of 
wmrm  grass  and  beautiful  wild  weeds  and  flowers- 
There  will  be  orchid  gardens  in  my  world  with 
various  fruit  trees  in  which  anijnals  can  delight 
on.    If  I  were  to  be  Mother  Nature  the  world  will 
be  rich  with  gifts  from  the  soil  aj\d  sky.  The 
season  will  come  at  their  intervals.    There  will 
be  showers  in  winter,  burgeoning  flowers,  fruits 
and  other  plants  in  spring.    The  suinmer  warmth 
will  deliver  the  harvest.   The  beaches  dance  with 
delight  as  the  people  splash  aid  celebrate.  The 
effulgence  of  the  sun  will  shine  the  warm  blue  sky 
through  my  paradise  and  at  dark  the  brilliant  stars 
shall  twinkle.    The  halo  of  smiling  Moon  shall 
reach  the  famlands,  forest,  prairries  and  waters 
in  my  beautiful  paradise,    I  won^t  tolerate  with 
evil.    It  shall  not  dwell  in  this  paradise  of  mine. 
People  and  animals  will  be  each  others  friends. 
They  will  help  one  another.    If  there  is  any  conflict 
betweeii  the  living  creatures  Til  handle  the  situation 
with  love  and  understanding,  which  all  of  the  creatures 
share  in  this  friendly  world-    The  beings  in  my  world 
will  be  immortal  and  can  change  into  whatever 
creatures  they  wish  to  be.    There  will  be  no  ruses 
because  my  people  and  animals  shall  remain  innocent; 
aware  but  innocent,  friendly  and  kind.    Every  living 
creature  shall  work  for  their  food.    People  will  plow 
the  fields.   Busy  cities  aikl  industries  shall  not 
exist.    There  will  be  a  fixed  pop>xlation.    No  more, 

As  Mother  Nature,  my  duties  will  be  to  regulate 
and  control  the  seasons »  beings  and  plants  in  my 
paradise  and  keep  the  living  creatures  happy  and  well. 

Topic  3,    Think  of  a  personal  experience  that  has  in  some 
way  changed  your  life:    a  particular  event;  a 
person;  a  place  you  have  visited;  a  book  you 
have  read;  a  film,  television  show,  or  play 
you  have  seen.    Describe  the  experience  and 
explain  why  it  was  an  important  one  for  you. 
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Low-Range  Schls.     Mid -Range  Schls.     High-Range  Schls. 
Score     No.        Percent      No.        Percent  No,  Percent. 


0 

1 

5 

0 

1 

2 

1 

2 

11 

0 

1 

2 

-? 

5 

16 

3 

10 

1 

2 

3 

5 

26 

8 

27 

5 

12 

4 

5 

26 

5 

17 

8 

19 

5 

1 

5 

7 

23 

11 

26 

6 

1 

S 

3 

10 

13 

30 

"7 

I 

5 

3 

10 

1 

2 

8 

0 

1 

3 

2 

5 

9 

0_ 

__0 

Jl 

TCTTAL: 

19 

30 

43 

Sarrple  Papers 
Low  -  Score  2 
Dear  school 

Nothing  has  rcaly    changed  my  life  except  for 
sone  funny  looking  plants  I  fo^.one  day  well  I  took 
it  to  my  friends  house  we  looked  it  over  for  a  few 
miJtites  and  finaly  decided  it  had  to  be  weed  so  we 
dried  it  out  awi  then  went  down  to 

Lower  Half  -  Score  4 

Brother  has  definatly  changed  my  life.    If  it 
^sn't    for  him  I  would  probably  not  be  so  mean  and  be 

able  to  trust  people  more.  u„«.e-^  mm 

All  nty  life  he  has  teased  me  and  haras ed  me  and 
I  had  to  Witch  what  I  $tid  or  did  to  keep  him  of  my 

"**'He  has  always  done  better  than  me  in  ?;<ath  and 
English  but  has  such  a  poor  imge  of  himself  that  he 
is  iealous  of  n»e.  .  , 

I  can  never  trust  my  brother  because  he  *l«>ys 
plays  tricks  on  me  and  I  don't  know  when  he  will  pull 
the' next  trick  on  me.  ,  ♦u,.  voe 

He  has  also  changed  my  life  in  a  way  that  has 
helped  me  get  along  better.    He  has  always  helped 
me  with  my  math  and  English.    If  it  wasn't  for  hun  I 
wouldn't  be  where  I  m  today  in  school. 

When  I  was  a  little  kid  my  brother  would  go  on 
talking  forever.    When  I  wanted  to  talk  he  would  go 
^  iS*no?  let  ne.    After  awile  I  ^Id  have  to  screan. 
"It  is  my  turn  to  talk  now."   Then  my  mother  vould  sa> , 
"It  is  Kevins  turn  to  talk  now,"  and  then  my  brother 
would  get  mad  at  me.    I  think  that  is  whv  T  have 
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trouble  saying  things  that  don't  sound  stupid  all  the 
tiine. 

Upper  Half  •  Score  6 

The  Turning  Point 

Aside  fron  the  everyday  normal  routine  of  life, 
there  are  things  that  interest  roe.    One  of  these 
"hobbies'*  are  reading  books,  especially  science 
fiction  and  fantasy  types,    I  must  of  read  over 
fifty  books  of  this  type  a  year,  but  there's  one 
book  that  stands  out  among  all  that  I've  read. 
The  books  title  is  'The  Thomas  Covenant  Trilogy.'' 
This  book  has  a  great  emotional  impact.  Remarkably, 
it  inscribed  a  big  mark  on  how  I  feel  and  react  in 
life  toward  my  environment-    This  book  made  me  more 
aware  of  my  surrounding  and  made  me  understand  loneli- 
ness. 

The  plot  of  the  story  is  a  leper  >fho  goes 
through  a  world  of  wonder  where  every  item  is 
essentially  iiiportant;  almost  alive.    The  words  that 
was  arranged  in  such  a  fashion  formed  a  description. 
This  description  got  absorbed  into  me.    I  actually 
felt  what  the  main  character  felt.    I  started  to 
choose  sides  on  who  I  want  most  to  survive.  The 
everchanging  story  got  me  involved.    It  gave  new 
ideas  and  wonders  about  the  normal  things  around  me. 
It  made  me  more  aware  of  my  surroundings  and  to  appre- 
ciate  things  of  luxury  instead  of  taking  it  for  granted. 

This  was  only  one  thing  I  got  from  the  book.  The 
book  also  made  me  understand  the  feeling  of  loneliness. 
While  reading  and  getting  involved,  I  actually  got  the 
feeling  of  the  main  character,    I  knew  what  it  was  like 
by  using  the  character  as  an  ©cample.   This  made  me 
react  differently  toward  people  who  are  usually  left 
out  of  activities. 

The  main  thing  is  that  books  can  change  your  whole 
perspective,  feelings,  and  understanding  of  things  in 
your  life. 

High  -  Score  8 

The  Experience  of  }^  Life 

When  ray  ninth  birthday  came  around,  one  of  the 
gifts  I  received  frcn  my  parents  was  to  take  ballet ^ 
lessons  at  our  apartment  complexes  recreation  building. 
Although  I  was  to  only  take  one  class  every  two  weeks, 
I  was  very  excited.    >V  dance  teacher  called  my  mother 
after  my  first  lesson  "saying  I  had  potential  and  to 
please  keep  up  my  lessons.    Of  course,  I  just  re- 
cently discovered  this  phone  call.    My  teacher's 
husband  was  the  artistic  director  of  the  local  ballet 
company  and  he  was  holdinc  auditions  for  a  new  dance 
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school"  that  would  be  affiliated  with  this  company. 
After  the  audition  and  my  first  class  with  him,  he 
told  my  parents  that  I  was  awarded  a  scholarship. 

Ever  since  that  scholarship  I've  seriously 
been  dancing,  taking  one  to  two  lessons  each  day. 
I  had  never  known  that  I  had  enough  talent  to  be  a 
ballet  dancer  until  my  scholarship;  and  if  it  hadn  t 
occurred,  I  might  never  have  continued  doing  what  I 
love  most--dancing. 

Topic  4.    While  visiting  in  your  grandmother's  house  you 
find  a  large  trunk.    Propped  on  a  chair  beside 
the  trunk  is  a  card,  and  on  it  in  your  grand- 
mother's handwriting  are  the  words,  "To  my  only 
grandchild:    Inside  this  trunk  you  will  find 
what  you  have  always  wanted." 
Describe  the  gift  and  hof  ytxi  will  use  it, 
Low-Range  Schls.     >^id -Range  Schls      High-Range  ScMs, 

Score      No.       Percent        No.        Percent  No^^  ^jrccrX 


0  1 


4  14  1 


114  0 

2  6  25  3 

3  3  13 


2  4 
11  2  4 

2  7  9  19 


4  8  33  8 

5  2  8 


28  U  23 


6  2  8 

7  1  4 

S  0 

9  0 


5             18  6  13 

5            18  11  23 

14  3  6 

3             11  3  6 


0 

TOTAL  24  28 


0 

48 


Sarple  Pcrpera 
Low  -  Score  2 

Inside  th<^  trunk  their  ^vas  three  pounds  of  tie 
weed     So  we  had  a  party  the  next  night  smoke 
five-OZ^rdrank  three^kegs  their  -as  tw«  hundred 
people  at  ray  party.    The  party.got  busted  «  three 
0  clock  in  thTmoming.    TTie  pigs  serched  the  house 
and  foind  the  weed  i  told  than  it  was  my  grandmother  s 
and  they  took  her  away. 

Lower  Half  -  Score  4 

In  the  trunk  was  a  bunch  of  twenty's,  ten's 
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and  fives.  When  I  was  down  counting  all  the  ironev- 
it  came  out  to  4,000,000  dollars,  I  wouiKiered  why 
she  left  it  to  me  in  her  will.  I  was  very  sad  be- 
cause it  made  me  rember  of  all  the  good  times. 

I  got  up  and  told  my  man  and  dad  about  it.  They 
told  me  to  put  it  in  the'  bank.    I  thought  it  was  the 
best  idea  but  I  kept  1,000,000  dollars  of  it. 

With  the  1,000,000  dollars  I  paid  of  the  bills 
like  telephone.  Electric,  etc.    I  then  bought  a 
motar  home  for  15,960  and  a  Jaguar  for  16,000  for 
my  selk.    I  also  gave  11,000  to  my  favorite  friend. 
I  thought  we  shoud  by  a  new  house  but  my  parents 
said  "not  yet." 

We  went  to  Lake  Tahoe  that  sumer  so  mv  parents 

could  gamble  some.    Will  they  gambled  I  went  to  the 
arcade  and  spent  a  bundle  of  money.   After  that  we 
took  a  trip  around  the  world. 

Upper  Half  -  Score  6 

Slowly  I  opened  the  trunk  and  was  amazed  at  what 
I  saw.    All  I  could  see  was  a  bright,  gleaming  green 
light  that  was  radiating  from  a  small  stone.  Around 
the  object  was  the  red,  silk  interior  of  the  trunk. 
I  did  not  understai^  why  ray  grandmother  vculd  want 
to  give  me  this  stone.    I  reached  down  and  picked 
the  emerald  colored  stone  up.    I  felt  the  sudrien 
surge  of  electrical  power  vibrating  through  my  body. 
My  brain  began  to  think  of  incredible  solutions  to 
problems  that  had  never  been  solved  before.    I  then 
realized  that  I  could  solve  the  world's  problems  with 
the  gift  that  my  grandmother  left  me. 

High  •  Score  8 

'•What  could  be  inside  the  trunk  that  Grand- 
mother would  have  given  to  me?"  I  asked  myself. 

Slowly  I  opened  the  old  creaky  top  to  the 
trunk.    My  h^art  was  pounding  with  excitement  as  I 
peered  into  the  trunk. 

'Magazines,"  I  said  aloud  disapointedly.  There 
niist  have  been  at  least  thirty  magazines  covering  the 
top  half  of  the  trunk.    I  started  taking  the  magazines 
out  hoping  there  would  be  something  more  valuable 
underneath. 

After  all  the  magazines  had  been  removed  my 
eyes  set  iqpon  two  boxes.    Both  were  rather  small  and 
black. 

I  carefully  reached  for  the  box  on  the  left  and 
opened  it.    Looking  back  up  at  me  were  two  shiny 
stones  set  in  earrings.    They  were  absolutely  beauti- 
ful!   In  the  box  on  the  right  was  a  huge  shiny  stone 
in  a  necklace.    There  was  a  little  card  attatched  to 
the  box  containing  the  necklace.    On  it  read  'These 
are  real  diamonds  Honey.    The  only  part  missing  is  the 
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ring  which  Rich  is  giviiig  you.   Con'gratulations  my 

heard  a  noise  behiivJ  tne  and  quickly  turned 
around.    There  was  Rich  with  ^^^I^^i" 

?iS*tAin  sTve  always  «nted»-di^nds  and  Rich. 
ISd  now  I  had  them  both.    I  would  treasure  these 
precious  things  always.  . 

•Thanks  Grandmother,"  I  wispereu. 

Tcvia  5.    Think  of  a  friend,  real  or  imaginary ,  that  you  had 
Tcp^cS.    ^STyou  were  younger.    Describe  something  you  a^ 
this  friend  did  together.    Try  to  show  your  reader 
the  kind  of  person  this  friend  was  and  why  you 
chose  this  person  as  a  friend. 

Low-Range  Schls .     Mid-Range  Schls .     High-Ran?e  Schls , 
Score   No.        P^rr^nt      No.        Percent  No.^  Percent 

0  0 

1  1 

2  5 

3  4 

4  9 

5  3 

6  3 

7  1 
S  0 
9  0 

TOTAL  26 


0 

0 

4 

1 

3 

1 

2 

19 

4 

14 

3 

7 

15 

7 

24 

5 

11 

35 

5 

17 

16 

36 

12 

6 

21 

4 

9 

12 

3 

10 

9 

20 

4 

2 

7 

1 

2 

0 

5 

7 

1 

3 

2 

5 

29 

44 

Sample  "Papers 
Low  -  Score  2 

My  Friend 

Me  and  ray  friend  use  to  have  lots  of  f^-  W« 
played  games  like  Scrabble.  Boggle       etc^  When 
the  bell  rang  for  recess.   We  always  played,  tether- 
ball,  on  the  monkey  bars,  and  on  the  swings.  At 
lunch  we  would  do  inich  the  same  things.   He  would 
aSSys  play  jokes  on  people.    For  instance,  this 
ri^l^got  so  iad  at  hlTbecause  at  lunch  he  would 
fined  up  a  squirt  gun  with  milk  and  squirt  milk 
at  her.   He  hardly  ever  got  mad  at  me 

In  May  and  June  there  where  water  balloons 
r;erywhere,  but  there  was  one  problw  we  didn  t 
havTiv.   SO  we  usually  ^^f^^^^^^l^S'^^r 
went  to  Alum  Rock  Park  we  aLuost  . 
way  back,  but  we  did  and  that  how  I  m  able  to  write 

this  paper. 
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Lever  Half  -  Score  4 


>V  Friend 

When  r  use  to  live  in  >tountai;i  View  I  had  a  true 
friend  njmed  Ton.    We  were  best  friends  since  I  can 
rerwi4>er.    When  I  was  six  my  friend  and  I  used  to  play 
liJce  we  were  in  a  war.   All  the  kids  in  the  block 
*ould  play,  but  me  and  Ton  were  the  best  shot.  We 
vwuld  hide  in  the  most  awkward  places,  thats  why  we 
were  the  best  shot.   When  I  was  seven  we  got  a  pool 
put  in  OUT  backyard.   When  the  first  trucks  came 
Tan  and  I  were  watching  T.V.    Tom  said  he  heard  the 
trucks,  r  was  so  excited  I  ran  to  the  door  and  instead 
of  the  door  I  went  flying  through  the  window  on  the 
side  of  the  door.    After  the  pool  was  half  completed 
we  used  to  run  dom  in  the  pool  and  back  up  the  other 
side. 

When  we  were  around  nine  we  used  to  get  big 
football  games  going  in  the  street.  Even  the  big  kids 
would  play.   We  used  to  also  go  and  play  golf  at 
Sunnyvale  nuniciple  golf  coarse.   We  eot  into  some 
pretty  radicle  games. 

Tom  and  I  used  to  go  and  catch  lizards  at  saint 
Francis  High  school.    We  cought  blue  bellies  and 
Ali gator  lizards  n*inly. 

Upper  Half  -  Score  6 

I  sat  back  to  in  my  recliner,  to  relax.    My  mind 
skipped  back  through  the  years,  which  seaned  centuries, 
to  the  gentle  world  of  my  youth.   When  living  was  sweet, 
no  worries  and  no  cares. 

I  remember  the  old  oak  tree,  where  dad  had  help 
Mike  and  I  build  the  fortress  of  our  dreams.  Mike 
v«s  quiet  around  people,  even  his  own  family,  but  when 
we  were  alone,  with  the  robins  sweet  song  and  the 
squirels  nimble  scampers,  Mike  was  a  bold  as' the 
most  voictourous  men. 

He  told  hw  he  would  in  the  future,  rich,  power- 
ful, and  proud.    Even  nw  I  can  visualize  him,  with  his 
sparkling  blue-green  eyes  and  hair  the  color  of  wheat 
just  about  to  be  reaped. 

Though  he  lived  two  miles  from  me,  we  were 
always  together.    Once  one  summer,  \Aen  the  sun  was 
but  rising,  we  "borrowed"  old  Mrs.  Wilsons  gentle  bay. 
She  used  the  horse  only  for  going  into  town  and  to 
chirch.   We  rode  to  our  hidden  and  secret  lake,  at 
least  we  thought  it  was  secret.   As  we  arrived,  to 
see  if  the  fishing  was  good,  it  began  to  sprinkle. 
It  was  odd  to  feel  rain  in  the  simmer,  but  it  was  a 
short  shower,  and  created  the  most  beautiful  rainbow, 
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over  the  lake  it  sparkled  and  glijinered. 

As  the  sun  rose  to  zenith,  we  heard  the  crackle 
of  twigs  behiTKi  us.   We  lumped  to  out  feet,  and  raced 
to  the  feeding  bay.    Jus'c  when  we  were  going  to  ride- 
off,  m/  father  shouted  to  us  •'Hold  up,  boys  its  only 
me,''  with  a  laugh. 

He  was  nice  about  it,  but  made  us  tell  old 
Mrs.  liaison  that  we  were  deeply  sorry,  though  we 
weren't. 

Ohh,  what  a  long  time  ago!   When  things  were 
siinplistic  and  I  was  so  inoccent  and  yoing. 

High  -  Score  8 

Rae  and  I  were  the  saine  age.  I  never  knew  her 
birthday,  but  she  always  seemed  to  keep  up  with 
me.    She  was  my  equal  when  it  came  to  heipbt  and 
we  never  fought.  Rae  had  black  ?uir  and  brown  eyes ; 
my  opposite  but  imaginary  friends  always  get  along. 

One  of  our  favorite  activities  was  to  make 
ourselves  a  part  of  the  latest  movie  we  had  seen. 
At  our  age  they  were  almost  all  put  out  by  Walt 
Disney,  but  our  favorites  were  the  Wizard  of  Or 
and  Willie  Wonka  and  the  Chocolate  Factory. 

The  night  after  we  had  just  seen  Willie  Wonka 
and  the  Oiocolate  Factory  for  the  first  time,  we 
immediately  began  on  our  journey  through  that 
magnificent  factory.    We  lived  through  each  scene, 
keeping  the  basic  format  but  naturally  altering 
certain  parts  so  that  we  fit  in  more  easily.  Then, 
of  course,  we  made  up  entirely  new  plots  to  add 
spice  to  the  story  and  make  it  even  more  enjoyable. 

We  eventually  became  the  heroines,  along  with 
Charlie  the  original  hero,  after  fighting  the 
vicious  knids  to  save  the  oantpM.    loonpas.  Well, 
Mr.  Wonka  did  help  a  little,  hatting  them  with  his 
cane  and  generally  putting  them  out  of  conmission. 

In  the  end  we  becane  part  owners  of  the 
chocolate  factory  to  visit  whenever  we  please. 
Mr.  Wonka  had  another  glass  elevator  built  for 
us  so  that  we  could  go  anywhere  wt  pleased  in  a 
matter  of  seconds.    Did  we  ever  maice  use  of  tiiat! 
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Tovic  6.    The  school  newspaper  has  asked  students  to 
submit  suggestions  about  how  to  iiqprove  the 
school.    Think  of  ONE  and  only  one  problem 
which  you  would  like  to  see  solved  to  make 
life  at  school  better  for  you  and  others, 
In  t  letter  to  the  school  newspaper  describe 
the  problem  and  tell  how  you  would  like  to 
see  it  solved. 


Low-Range  Schls.     Mid-Range  Schls.     High-Range  Schls. 


ore 

No. 

Percent 

No.- 

Percent 

No. 

Percent 

0 

1 

5 

0 

1 

3 

1 

1 

S 

0 

1 

3 

2 

3 

16 

1 

4 

2 

5 

3 

3 

16 

3 

11 

6 

15 

4 

6 

32 

S 

19 

10 

26 

S 

2 

11 

6 

22 

7 

18 

6 

3 

16 

3 

11 

5 

13 

7 

0 

5 

19 

3 

8 

8 

0 

4 

15 

4 

10 

9 

0 

0 

0 

TOTAL  19 

27 

39 

Sarrpl€  Papen 
Low  -  Score  2 

The  w«y  the  school  could  approve  is  by  having  everybody 
being  friends  beacouse  there  are  to  many  people  in  this 
school  that  are  prejudice  and  I  think  that  it  should 
not  be  that  way, everybody  should  be  friends  it  doesn't 
matter  what  race  you  are,  and  also  the  teachers  you 
can  tell  that  soRe  are  prejudice  beacouse  the  way  they 
treat  you  and  the  way  you  see  them  treat  the  other 
pupils.   And  maybe  that  will  at  least  stop  scfne  fights 
against  different  races  and  it  will  stop  riots,  and 
it  might  bring  people  closer  to  each  other,  like 
walking  by  and  say  I  hate  that  person  because  shes 
black,    It  shouldn't  be  that  way.    Everybody  should 
like  each  other  its  not  good  to  be  prejudice,  you 
would  get  more  freinds  if  you  werent  prejudice  and 
I  think  that  you  would  have  a  better  life  kno**ing  that 
you  can  depend  on  any  of  your  friends.    And  I  think 
thats  the  way  it  should  be.    Its  up  to  the  people  the 
way  they  want  to  be  because  its  there  life  and  if  they 
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want  to  be  prejudice  let  then  its  than  that  are  not 
going  to  have  very  many  friends  when  they  need  some- 
thing.   So  it  is  good  for  you  not  to  be  prejudice. 
So  be  friends  with  everyone,  you'll  get  more  out  of 
life  to  have  friends. 

Lower  Half  -  Score  4 

The  one  problem  I  think  our  school  has,  is  not 
enough  people  getting  in  the  'spirit'  of  things; 
mainly  days  like.  Toga  Day'  and  the  work  parties  for 
Spirit  Week. 

Oh  days  like  toga  day,  hat  day,  etc.  a  lot  of 
people  did  get  into  the  spirit  of  things— but  not 
enough. 

At  the  work  parties  for  "Spirit  Week,'  I  was  sur- 
prised at  the  number  of  people  that  didn't  show  up. 
At  least  I  loiow  that  is  true  for  the  sophomore  class. 
The  work  parties  are  arranged  at  meetings  to  have  it 
at  sooeone's  house  when  everyone  is  available,  ftit 
a  lot  of  times  not  enough  people  show  up. 

I  think  this  problem  can  be  solved  by  pitting 
more  stress  in  the  idea's  for  spirit  days  and  spirit 
vork  parties.    It  could  also  help  by  having  more 
meetings  to  prepare  for  this.   More  people  should 
help  spread  the  word  (sane  people  may  not  know 
about  it),  and  get  involved. 

Upper  Half  -  Score  6 

Have  you  ever  had  the  problem  of  wanting  to  transfer 
schools  S  found  out  you  couldn't  for  some  dumb  reason? 
Chances  are  that  you  probably  havn't  since  only  about 
951  of  the  kids  ever  want  to  transfer,  but  it  is  some- 
thing you  should  be  concerned  about. 

Most  people  don't  teww  that  it  is  a  law  of  Cali- 
fornia that  your  parents  can  choose  to  send  you  to  any 
public  school  they  choose.    So  instead  you  find  yoir 
self  calling  the  District  Office  and  asking  why  you 
can't  transfer  or  going  in  to  see  the  deens  or  the 
principal. 

Sane  people  just  lie  about  there  address,  but 
most  people  don't.    Instead,  they  try  and  do  it  law- 
fully. 

But  Wien  you 'do  talk  to  someone  of  that  sort 
they  will  tell  you  "I'm  sorry,  but,  unless  you  have 
a  health  or  mental  problem,  you  nust  go  to  the  school 
in  the  boundry  of  wfdch  ytju  live."  They  say  that 
they  need  to  have  these  rules  in  order  to  control 
the  population  in  the  schools  but  it  is  just  not 
true.    They  have  no  problem  with  the  amount  of 
students;  they  have  a  problem  with  the  teachers. 
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So  I  guess  what  you  realy  need  to  be  concerned 
about  is  to  know  your  rights  5  don't  let  people  rip 
you  off  for  no  reason. 

High  •  Score  8 

April  22,  1980 

To  the  editor, 

I  am  responding  to  the  article  concerning  school 
problems  which  appeared  in  the  April  17th  edition  of 

the    I  feel  that  the  athletic  program  at 

snould  be  iinprored.    Sports  at 
are  suffering  from  a  lack  of  participation  by  students 
aivi  teachers. 

The  main  reason  ^y  students  don't  participate 
is  apathy:    they  just  don't  care.   They  don't  care 
about  schoolwork,  athletics,  or  even  their  own  bodies. 
Teachers  don't  take  an  active  role  in  athletics  by 
coaching  or  helping  out  because  they  feel  the  stixients 
don't  appreciate  them  or  it  just  takes  too  much  time. 
These  problems  are  unfortunate,  but  true.  Consequently, 
the  morale  here  at  is  suffering. 

I  feel  that  if  athletics  were  publicized  and 
incentives  were  given  to  try  out  for  a  team,  this 
problem  could  be  solved.    If  athletics  soared  to  a 
new  level  of  participation  and  enthusiasm,  many  other 
school  problems  would  be  solved.    Students  would  work 
hard  on  their  schoolwork  so  that  they  could  have  the 
opportunity  to  participate.    Drug  use  would  be  lowered 
because  drugs  and  athletics  don't  mix.   Also,  school 
spirit  would  rise  and  teachers  would  be  more  enthusiastic 
about  coming  to  school. 

Sincerely  yours, 
John  Doe 

Topic  7.    Bnploymcnt  agencies  can  be  very  helpful  in  finding 
you  a  job  if  they  know  what  job  you  want  and  are 
qualified  for.    Write  a  letter  to  The  .ABC  En?)loy- 
ment  Agency,  204  East  Main  Street,  Bay  City,  C-^ 
94606.    Explain  what  job  you  would  like  and  what 
your  qualifications  are.    Explain  the  reasons 
you  think  you  should  be  hired  and  give  exaiT^)les 
of  your  strengths.    Include  references. 
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Low-Rancre  Schls.     Mid-Ran.ee  Schls.     High-Range  Schls. 
Score     No.       "^Pgrcent      No.        Percent      No.  PeTcent_ 

0  0  14  0 

1  Q  2  8  2  4 

2  7  37  2  8  8  17 

3  5  26  3  12  5  U 

4  3  16  6  24  13  28 

5  2  11  4  16  3  6 

6  2  11  4  16  12  26 

7  0  3  12  1  2 

8  0  0  1  2 

9  0  0  2  4 

TCTTAL    19  25  47 

Sarrple  Papers 
Low  -  Score  2 

April  22 

ABC  Employment  Agency 
204  East  Main  Street 
Bay  City,  CA  94606 

I'm  really  interesting  about  finding  a  job.  I'tti 

in   high  School  and  after  I  Graduate  I  like 

to  go  to  College.    Which  is  four  years  of  college. 

I  hope  if  you  can  help  out  to  finding  my  job.  My 

mjor's  would  be  Mecanical  noise.    If  I  Graduate  college 

and  get  the  job  how  much  do  I  get  paid? 

Do  I  get  more  money  if  I  Graduate  College? 

My  suport  is  Graduating  College  and  have  d  30b  and 

settle  my  life. 

Is  it  difficut  to  have  a  job? 

And  give  me  some  informatiens  of  how  to  get  in  and  how 
niich  do  we  have  to  vork  and  so  on. 
I  can  speak  English  but  not  much  of. 
Because  it's  my  second  language. 
I  cane  to  this  country  about  a  four  years  ago.    And  I 
really  enjoying  living  here.    One  thing  trouble  is  my 
English  is  not  complete  and  easy. 
But  I  can  study  hard  and  go  to  college. 
Send  me  some  informations  of  job  and 
Thank  you  for  reading. 
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Thank  you  again* 


Your  truely, 


Lower  Half  •  Score  4 

ABC  EnqDloyment  Aency. 
204  Eeast  Main  Street 
Bay  City,  Cml.  94601 

To  whom  this  may  concern: 
I  think  and  have  dreajned  about  being  a  stewerdess.  I 
think  that  I  would  make  an  excelmt  one.    I  Love  to  be 
with  people,  ytxmg  and  old.    In  high  school  I  have 
taken  4  years  of  language.    I  Love  to  fly.    And  I  like 
to  be  on  the  go  all  the  tijoe.    I  have  never  liked  just 
sitting  aroun. 

I  would  like  to  fly  with  a  world  wide  airline  so 
that  r  may  find  out  as  many  cultures  on  all  the  people  of 
the  world  as  I  can.    I  have  had  some  conections  with 
people.    I  started  out  when  I  was  twelve  years  old 
babysitting.    I  babysat  until  I  was  fifteen  the  got 
little  jobs  like  being  a  waitress  and  working  in 
department  stores.    They  might  no  be  luxeryes  but  I 
did  it  all  so  that  I  coiald  learn  more  about  people. 

Another  job  I  would  like  to  do  to  help  make 
it  a  little  easyer  for  being  a  flight  atendant  is  to  be 
a  bar  tender.    I  know  alot  of  peoole  like  to  drind  when 
they  are  on  a  flight  halfway  around  the  world. 

Well  I  hope  you  can  find  me  something  to  fill  my 
lifes  dream. 

SiJKcrly 

Uoper  Half  •  Score  6 

ABC  Brnploymcnt  Agency  Jeff  L  

204  East  'Main  St.  1405  Sequoia  dr. 

Bay  City  CA   94606  Gilroy  CA  4/23/80 

Dear  Sir, 

I  am  writing  this  letter  to  inquire  about  employment 
from  your  agency.    I  am  a  licensed  and  qualified  custom 
car  painter  and  body  maii.    I  i^uld  like  to  be  en^loyed 
in  an  established  and  prosperous  business.    The  reason 
I  ask  this  is  because  I  have  had  nothing  but  bad  luck 
with  new  body  shops  that  have  just  started  out.  I 
think  that  any  custom  shop  would  be  glad  to  hire  me. 
I  have  had  many  hours  of  classroom  and  on  the  job 
training  at  Gavilon  College  in  Gilroy.    I  also  have  a 
complete  history  of  ray^  work  that  I'm  sure  an  employer 
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would  be  very  interested  in,    I  have  worked  for  two 
body  shops  in  town  that  I  will  put  down  for  references, 
Bob  Hogue's  body  shop  716  Leanesly  Rd-842-5671  and 
Ridy's  body  shop  441  Colmbet  ave. -842-6699,  If 
you  need  any  more  information  please  contact  roe. 

Sincerly, 


Jeff  L 


High  -  Score  8 


10712  F   r>r. 

Cupertino,  CA  95014 
April  22,  1980 


The  ABC  Bnployraent  Agency 
204  East  Main  St. 
Bay  City,  CA  94606 

Dear  Sir, 

name  is  M  P  and  I  am  a  student  at   

High  School .    I  aiTF  years  old  and  I  have  lived  in 
Cupertino  for  14  years. 

I  am  looking  for  a  part-time  job.    I  im  interested 
in  being  a  bus-boy  because  I  have  ha.   scxne  experience. 
I  feel  by  having  this  experience  it  makes  me  qualified 
to  have  a  position  as  a  bus -bey.    Some  of  my  other  quali- 
fications are  that  I  enioy  being  with  and  around  people, 
I  am  a  hard  worker,  and  I  always  try  to  do  the  best  that 

I  possibly  can.  ,      .  ,   ,  ,  j 

I  am  listing  three  people  that  I  feel  know  me  and 
how  I  work.    These  people  are  my  past  enployers. 


 Restiirant 

c/o  

10752        "  Rd. 
Cupertino,  Ca.  95014 

Mrs.  . 

10777  gr. 
Cupertino,  Ca.  94014 


197  55  Place 
Cupertino,  U.  95014 

I  hope  that  you  will  consider  my  application  and 
trv  to  find  a  job  for  me.    Thank  you  for  spending  the 
time  to  read  my  letter  and  I  hope  to  hear  a  response 
from  you  in  the  near  furture.  Sincerely  yours, 


"J  \J  <' 
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Topic  8.    You  have  been  sent  a  bill  diarging  ycxj  for  tv,-o 
record  albums  you  did  not  purchase.    Write  a 
letter  explaining  the  problem  so  that  scmeone 
in  the  billing  departjnent  can  correct  the  error. 

The  bill  is  from  Melodie  Record  Club,  444  Tune 
Street,  Mashville,  Tenn.  98056. 

The  records  charged  to  your  account  are  Kiss 
Prt8ent8  Opera  EighlightB  and  Led  Zepplin 
Country  Bite, 

Pretend  your  name  arid  address  are  Chris  Brov^-n, 
456  Main  Street,  Bay  City,  CA   92001.  Your 
account  number  is  35-800-21. 

Lc/f-RanKe  Schls.     Mid-Rantje  Schls.     High-Range  Schls, 
Score    No.        Percent      No.        Percent      No.  Percent 

0  0  2  9  0 

10  14  12 

2  2  12  0  3  8 

3  1  6  1  4  9  22 

4  7  41  5  22  12  30 

5  5  29  4  17  6  15 

6  1  6  3  13  0 

7  1  6  5  22  S  13 

8  0  2  9  4  10 

9  0  0  0 

TOTAL  17  23  40 


Sarnple  Papers 
Low  -  Score  2 

Dear  Sir 

I  un  writting  to  you  about  two  albulms  I  was 
charged  for.    The  albims  are  Kiss  Presents  Opera 
Highdights  and  Led  Zepplin  Country  Hits    but  I 
didn't  purchase  these  albums.    My  accont  number 
is  35-300-21.    I  hope  you  will  take  care  of  this 
problem, 

Chris  Brown 
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Lcjwer  Half  -  Score  4 


Dear  Sir, 

I  recieved  a  bill  charging  me  for  two  albums  I  did  not 
buv.    The  two  albums  were  Kiss  Presents  Opera  Highlights 
and  Led  Zenplin  Country  Hits.    I  checked  my  albums  and 
did  not  se'e  these  two  albuns  in  ray  collection.  The 
bill  I  recieved  came  from  444  -Hine  Street  Nashville 
Tenn.    98056    I  checked  ray  records  I  have  found  that 
there  has  been  NO  checked  mailed  to  Nashville  Tenne- 
sse.    The  only  thing  I  can  think  of  that  went  wrong 
was  that  your  computer  thought  that  I  was  saneone 
else.    I  would  be  happy  to  buy  these  two  albums,  but 
I  will  not  pay  the  bill  until  I  recieve  the  two 
albums ,  as  it  mav  result  in  another  mishappen  .1 
would  appreciate'  some  information  regarding  this  sub- 
ject.   I  would  also  like  to  know  what  went  on  5  how 
this  occured. 


Upper   Half  -  Score  6 


Melodie  Record  Club 
444  Tune  Street 
Nashville,  Tenn.  98056 

Dear  Sirs: 


456  >fain  Street 
Bay  City,  CA  92001 


My  name  is  Chris  Brown  and  I  am  writing  to  you  in 
complaint  to  a  recent  bill  which  was  sent  to  me  by  your 
store.    The  bill  claims  I  charged  two  record  albums 
which  is  not  true.    The  records  wert  Kiss  Presents 
Opera  Highlights  and  Led  Zepplin  CountTv  Hits.  I 
assune  there  nust  have  been  amistaxe  somewhere  in  your 
billing  department  which  triggered  the  bill  to  be  sent 
to  me.    I  would  be  veiv  appreciated  if  you  could 
scnehow  re-track  the  error  so  the  bill  can  be  voided, 
account  nLsnber  i$  35-800-21. 
Thank  you  very  nuch  for  your  cooperation  and  i 
am  happy  to  be  one  of  ycwr  customers. 

Sincerely, 
Chris  Brown 
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High  -  Score  8 

Chris  BrovvTi 
456  fiain  Street 
Bav  Citv,  Calif. 

92001 

Melodic  Record  Qub 
444  Tune  Street 
Nashville,  Term. 

98056 

4-29-80 

Dear  Sirs, 

I  have  been  billed  for  two  record  albums,  Kiss 
Presents  C^pera  Highlights    and  Led  Zepplin  Country 
Hits,  which  I  did  not  order  or  receive. 

name  is  Chris  Brown  and  my  account  number 
is  35-800-21.  If  you  check  your  records  I'm  sure 
that  you  will  find  the  mistake  and  clear  the  problen. 

Thank  you  for  your  cooperation. 

Sincerely, 

Chris  Brcfwn 


8. 0    Classroom  Practice  for  Writing  Coinpetency  Testing 

Writing  short  timed  essays  on  subjects  students  have  had  no 
opportunity  to  prepare  for  requires  scnewhat  different  skills 
from  those  needed  for  the  usual  school  writing  assignments.  It 
seems  only  reasonable,  therefore,  that  students  should  be  given 
some  training  in  these  skills  before  being  required  to  demonstrate 
their  conpetency  with  them. 

The  best  training  for  tests  of  this  sort  is  frequent  in- class 
practice  with  the  kinds  of  topics  v^ich  will  be  used  for  actual 
testing.    Students  can  learn  how  their  test  papers  are  judged  it 
they  also  learn  to  score  each  other's  papers  themselves  using  the 
holistic  method. 

It  is  suggested,  however,  that  teachers  be  discouraged  from 
using  the  field-tested  prompts  included  in  this  study  for  class- 
room writing,  as  over-exposure  will  make  their  use  in  actual 
testing  invalid  for  scjne  students. 

In  place  of  the  tested  topics,  teachers  are  encouraged 
use  the  practice  topics  below  and  to  create  their  ovsn  similar 
topics.    These  topics  can  all  be  scored  according  to  the  scorm,: 
guides  on  pp.  35l-352above. 
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Santa  Clara  County  Writing  Assessment 


Practice  Topics 

1.  Think  of  someone  you  i^iould  like  to  be  v^ith  right  now. 
Describe  this  person  and  your  relationship  with  him  or 
her  in  sxich  a  way  that  your  reader  can  understand  your 
choice. 

2.  You  are  a  ver>'  small  girl  or  boy  being  dragged  along  by 
your  mother  or  father  in  a  large  crowd.    Describe  your' 
feelings,  and  tell  what  you  see,  hear,  smell  and  feel 
in  as  much  detail  as  you  can.    Help  your  reader  imagine 
what  it  is  liJce  to  be  there. 

3.  If  you  could  change  places  with  someone  else,  who  would 
it  be?   The  person  you  write  about  can  be  living,  dead, 
drawn  fron  past  or  present,  from  books »  movies,  tele- 
vision, or  your  own  imagination.    Try  to  give  your 
reader  some  idea  of  what  it  would  be  like  to  be  that 
person,  and  of  why  that  life  appeals  to  you. 

4.  Most  of  us  have  objects  that  we  treasufe,  not  just 
because  of  their  value  in  money,  but  for  other  reasons. 
Describe  one  or  more  objects  which  are  iinportant  to  you. 
Explain  why  you  value  them. 

5.  Not  all  inventions  have  been  good  for  all  humanity. 
Name  one  invention  we  would  be  better  off  without, 
and  make  it  clear  why. 

6.  What  would  you.jlke  to  be  able  to  invent  or  create? 
Indicate  v^iat  need  your  invention  or  creation  would 
fill,  and  describe  or  explain  it  in  as  much  detail 
as  possible. 

7.  Yowr  school  newspaper  announcts  thar  U0,000  has  been 
given  to  your  schopl  to  mprove  extra -curricula^  programs 
such  as  art,  wusic,  drama,  sports,  field  trips,  and'^any- 
JJung  else  the  school  wishes  to  ap|>ly  it  to.   Write  a  * 
letter  to  the  school  newspaper  suggesting  ONE  thing  you 
think  the  school  should  do  with  some  or  all  of  the  money. 
Tell  .ciearly  hcv  the  money  ii^nild  be"  spent  and  give  reasons 
irfiy  yoli  think  money  should  be  spent  on  'the  suggestion. 

8.  ^  Wri't^  a  friendly  letter  to  a  '"pen  pal''  in  a?x)ther  country, 

telling  hiin'or^her  about  your  preparations  for  some 
holiday,  ^  Your  letter  should  be  as  •'news/'  as  possible 
because  your  friend  probably  does  not  know  how  you  do 
things  where  ^ou  live. 

9.  You  ordered  an  engraved  pen  from  Qualitv  Products;  Inc. , 
V    4S6  -Treat  St.,^  Bay  City,  CA   94012.*  The  pen  was  to  be  a 

giftn  And  was  to  have  the  name  Lee  4Acobs  engraved  on  it. 
When'^it  arrived,  it  was  engraved  Les  Jacobs.   Write  a 
letter  to  the  conpairy  e^qslaining  the  error  and  asking 
them  to  cort^ct  it.    Pretend  your  name  is  Chris  Brown 
and  your^address  is  JOS  Main  St.^  Richford,  CA  94018. 
Use  correct  business  letter  form. 
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You  have  heard  that  Biggies  Department  Store,  885  >iain 
Street,  Bay  City,  CA  94012  is  hiring  teenagers  for 
sunmer  work.   Write  a  letter  to  Ms.  Janet  Jones, 
Personnel  Manager,  applying  for  the  job.    Explain  what 
job  in  the  store  you  would  like  and  lAat  your  qualifi- 
cations are.    Explain  the  reasons  you  think  you  should 
be  hired  and  give  exaroles  of  your  strengths.  Pretend 
your  name  is  Qiris  Brown  and  your  address  is  208  Country 
Lane,  Pleasant  City,  CA  9Q843.    Include  references  and 
follow  correct  business  letter  fom. 
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Appendix  A 


Members  of  the  Santa  Clara  County  Writing  Committee 
Who  Assisted  in  Developing  the  Prompts 


Rosalie  Chako,  Redwood  Junior  High  School 
Saratoga  Elementary  District 

Dawn  Chase,  Westmont  High  School 

Campbell  Union  High  School  District 

Eleanor  Hester,  Blossom  Hill  Elementary  School 
Los  Gatos  Elementary  District 

Karen  Lilly,  Cupertino  High  School 

Fremont  Union  High  School  District 

James  Marshall,  Los  Altos  High  School 

Mountain  View-Los  Altos  Union 
High  School  District 

Irvin  Peckham,  Live  Oak  High  School 

Morgan  Hill  Unified  School  District 

Betsy  Reeves,  Sunnyvale  Elementary  School  District 

Elizabeth  Ruttencutter,  Department  of  Instruction 

Milpitas  Unified  School  District 

Mary  Williams,  Fair  Intermediate  School 

Franklin-McKinley  School  District 

Ellen  Wilson,  Los  Gatos  High  School 

Mountain  View-Los  Altos  Union 
High  School  District 
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Appendix  B 

Sample  Sizes  for  Testing  Topic  Perforrance 


Several  rules  of  thjmb  apply  to  deciding  what  nutkes  an  adequate 
sample  size.    There  are  some  statistical  miniJitnns  based  on  the 
need  to  produce  a  normal  distribution  of  scores:    these  vrould 
require  a  oininm  (for  liberal  rather  than  totally  rigorous 
procedures)  of  twenty  cases  to  a  cell .    In  the  study  yoi  are 
ureiertaking ,  this  %«juld  mean  twenty  sai^Jles  of  writing  on  each 
topic  fran  each  school  you  wish  to  generalize  aboit.   For  an 
accurate  and  comprehensive  diatriat  sampling,  you  would  need  to 
sample  each  schojl  in  the  district.   The  twenty  wwild  be  a 
minijTun  set  to  insure  statistical  significance  of  differences, 
no  natter  how  snail  a  particular  school  or  district  population. 

The  minimm,  however,  may  not  provide  a  rtprtatntativt  $mple, 
especially  of  a  large  school.   However,  careful  atratified 
random  $ampling  proctdureB  can  help  insure  that  the  twenty 
papers  on  each  topic  are  a  representative  performance  fran  a 
given  school  or  district.   The  miniflun  of  twenty  papers  per 
topic  %dthin  each  school  would  then  insure  reasonable  chance  of 
showing  statistically  significant  differences  if  real  differen- 
ces existed  between  topics  and  their  effects  on  students. 

You  have  two  schools  In  your  sample  which  have  poiwlations  of 
only  680  students  and  twelve  schools  with  fewer  than  1500 
students.   At  the  risk  of  gathering  a  slightly  less  represen- 
Utive  saanple,  for  these  schools  you  could  reduce  the  total  num- 
ber of  sanples  to  ten  per  topic  for  the  anallest  schools,  and 
fifteen  per  topic  for  the  schools  which  have  1000-1400  students. 
Statistically,  if  you  are  concerned  with  how  a  topic  performs 
within  a  district  (rather  than  within  a  school)  you  could 
actually  get  by  with  only  twenty  papers  per  topic  from  each 
di$triet.    The  problem  here,  of  course,  is  that  it  would  be  a 
difficult  task  to  choose  the  twenty  students  in  each  district 
who  would,  in  their  abilities,  represent  proportionally  the 
spread  of  abilities  in  the  district.    Given  all  the  factors  that 
affect  student  writing  perf orinance ,  including  individual  school 
programs,  I  would  reconmenl  stratified  sampling  within  schools, 
using  the  school  as  a  unit  of  measure  rather  than  the  whole  dis- 
trict, and  thjs  gathering  twenty  samples  per  topic  from  each 
school . 
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Whatever  your  decision,  these  figures  must  give  way  of  course 
to  the  ultijiate  costs.    But  rather  than  cutting  down  the  number 
of  samples  per  topic  to  reduce  costs,  I  would  strongly  advocate 
cutting  the  number  of  topics  tested.    You  could  spend  a  great 
deal  of  money  to  gather  only  two  or  three  or  five  samples  from 
each  school  or  district  for  each  topic,  only  to  find  that  the 
spread  of  ability  represented  was  totally  misleading.    Such  a 
study  might  cause  you  to  discard  good  topics  v*)ich  would  have 
discriminated  well  amonj^  good  writers  in  a  particular  school, 
for  instance,  sinply  because  only  poor  writers  were  among  the 
few  who  happened  to  write  on  that  topic  in  the  "good"  school  of 
the  district. 


Catharine  Keech 

Research  Assistant 

Bay  Area  Writing  Project 
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Appendix  C 


Teachers  Who  Participated  in  Scoring 
Writing  Sample  Essays 


Head  Reader 
Gerald  Csmp 

Bay  Area  Writing  Project 

Fremont  Union  High  School  District 


Table  Lexers 

Rosalie  Chaico 

Redwood  Junior  High  School 

Saratoga  Elementary  School 

Irvin  Peckham 

Live  Oak  High  School 

Morgan  Hill  Unified  School  District 


Readers 

Elizabeth  Ruch 
Independence  High  School 
East  Side  High  School  District 

James  Marshall 

Los  Altos  High  School 

Mountain  View-Los  Altos  Union  High  Schx)l  District 

George  Dailey 

Live  Oak  High  School 

Morgan  Hill  Unified  School  District 

E.  Maxine  Thcnpson 

Britton  Middle  School 

Morgan  Hill  Unified  School  District 


Valerie  Solaris 

Hirphy  Middle  School 

Morgan  Hill  Uhif ied  School  District 
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Diane  Wieden 

Landels  Elementary  SchcxDl 
Nbuntain  View  Schcx)l  District 

Mary  Ycxmg  Williams 
Fair  Intermediate  Schcx)l 
Franldin-McKinley  School  District 

Terry  Bernard 

Live  Oak  High  School 

Kfargan  Hill  Unified  School  District 

Greg  Guerin 

Gilroy  High  School 

Gilroy  Unified  School  District 

A.  Spain 

Live  Oak  High  School 

Morgan  Hill  Unified  School  District 

Eleanor  Hester 

Blossom  Hill  Elementary  School 

Los  Gatos  Elementary  School  District 

Bert  HDward 

Gilroy  High  School 

Gilroy  Unified  School  District 
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Chapter  A 


A   STUDY   OF  THE   CONSTRUCTION   OF   THE  MEANINGS 
OF  A  WRITING   PROMPT   BY    ITS   AUTHORS,  THE 
STUDENT   WRITERS,    AND  THE  RATERS 

Sandra  Murphy,    Karen   Carroll,    Charles  Kinzer, 

and  Ann  Robyns 


Part    I:  Overview   of   Goals   and   Procedures   of  the 


Part  II:  The  Writing  Prompt:  Differences  in 
Expectations  Among  Participants  in 
a  Writing  Test  Episode 


Part    III:      The  Writing  Prompt:    The   Process  of 
Interpret  at  ion   and  Performance  That 
It   Elicits   in   Student  Writers 


Study 


Karen  Ca 


rroll   and   Sandra  Murphy 


Charles  Kinz 
S 


er,  Karen  Carroll,  and 
andr a  Murphy 


Sandra  Murphy,    Karen   Carroll,  and 
Ann  Robyns 
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Chapter  4:      Part  I 

OVERVIEW  OF   GOALS   AND  PROCEDURES 
OF   THE  STUDY 

Karen   Carroll   and   Sandra  Murphy 

I n  troduction 

In   any  co.-n.n  un  i  c  a  t  i  on   situation,   one  participant 
,ay  .misconstrue   the  meaning  another   pa  r  t  i  c  i  pa  n  t   i  n  t  en  1  s 
to   "onvey.      Such     is  und  er  st  an  d  in  g   is  a   proole.  of  spe- 
cial  concern  within   the   context  of  assessiient. 

Assessment   frequently  occurs  in  educational  situa- 
tions,     one   such   situation  is   found   in  the 
setting.      Tns   type  of  assessment  common   to  the 
room  occurs  in   the   following   sequence:   an   oral   or  writ- 
ten quesrion  prompt  is  posed  by  the  teacner;  an   oral  or 
written   answer   is  constructed   by  the  student;   and  an 
oral  or   written  comment  on  the   student's   answer  is 
Tiven   by   the   teacher.      This  qu  e  s  t  i  on-an  5  w  er -comm  en  t 
format,    (cf.   Mshan.    1979  )   is  comparable   in  soiie 
resp'cts   to   large-scale   writing  assessment  episodes. 
MjltiDl-  opportunities   for  m i s comm un ic a t i on   exist  in 
both  of"tnese  assessment   situations.      However,  an 
important  distinction   exists  between   the   two  cases, 
i^ile   the   teacher  may  readily  correct  a  mismatch 
between  his/her   intended   meaning  and   the  student's 
response   in   tne   classroom  setting,    in   an   annual  large- 
st a  fe   a  s  s  e  s  sm  en  t  episode,   the  opportunity  for  correc- 
tion does  not  exist  once   the  student  has   received  the 
orompt   and  begins  to  write.     The  consequences  o f 
misconmunicatlon   in   the   latter   setting,    therefore,  are 
more  grave   and  lasting. 

Tne   situation  reported   here  is  a  writing  assess- 
m.nt  "eDisode."   Tne  method  of  assessment  used   in  the 
olrticular   "episode"   under   study  Involved   an  holistic 
appraisal   of  student  writing.*   Tne   issues   raised  by  the 

-gy.r~r^7r^ir^r:-r5~STssI~u?on  research     supported     by  the 
yfy  „^?      Institute  of   Education  under   Grant   No.   NIE  G- 
oS^U   to   tSe   Bay  A?ea  Writing   Project.    University  of 
?ar??ornJa.    Berkeley.    Leo   Ruth.   Writing  Assessment  Pro^ 

ilhe  ^iethod^'of  holistic  assessment  has  been  described 
K  M.nVis  (1931)  as  a  procedure  in  whicn  aultiple 
?Lde  'eva  -tors  concentrate  upon  forming  an  overall 
Jmoression  of  each  paper  relative  to  the  other  papers 
J.  ^v  have  read.  The  primary  objective  of  holistic 
lHo^ina  is  the  -valuation  of  a  student's  writing  rela- 
tive to   the  performance  of  th^t  student.'s  peers.  Myars 
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stJdy  are  pertinent  to  Tistnods  of  writing  assassTient  in 
gen  er 3l  . 


A  writing  assess. u ant  '^episode'*   consists  of  tne 
following  stages:    1)   the  initial  d  e  v  el  o  p.n  en  t  of  a 
proTjpt  by   test-makers    (inclujing   the   formulation  of  the 
wording   of  a   written  version);    2)    the   reading  of  the 
prompt    (including   initial   reading   and   subsequent  re- 
reading at  any  stage);    3)   the  writing  of  compositions 
in  response   to   the  prompt;   M)   the  development  of  a 
scoring   rubric   and;    5)   the   use  of  that  rubric  in  a 
scoring  session. 

Tnere  are   a  variety  of  participant  groups  and 
texts   involved   within   a  writing  assessment  episode:  1) 
prcTipt   authors,   who  design   a   written   paragraph  to  serve 
as   a   prompt;   2)   students,   who  write  compositions  in 
response   to   that  prompt;   and   3)   evaluators,   who  read 
and   score   the   student  compositions   (Sae   Figure  1). 


T«tt  H«k»r 

<Cff<icKvr  I 


3  ! 


.i  i 


/I 


f valuator 


ri|tur*  I.       r«rt  lel|»«nt«  mx\A  7«ttA  TnvnW*^  In  «  Vrltlnn  Afifi«>«««rf<t  fPUnrfy 


(1930)  describes  the  method  of  holistic  assessnent  as  a 
procedure  "using  an  overall  ifopression  to  rate  a  paper 
on  a  numerical   scale, (p.  1). 
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Tne   two  texts   identifieJ   in  Figure   1  are   the  writ- 
ing  proTipt,   and   the  written  co-nposition  prodused  by 
easn  student.* 

In   order   to   determine  whether   participants   in  a 
typical   writing   assessment  episode  construct  different 
interpretations  of  the   prooipt   3nd   form  different 
interpretations  of  the   assessment  task  in   general,  we 
made  arrangements  to   follow  the   progress  of  one  writing 
assessment  episode  and   to   record   th-   impressions  of  the 
participants  in   detail   at  several  key   points   along  the 
way.     These  points,   where   the   participants  interact 
with  texts  in   the  assessment,   are  indicated   by  the 
arrows   in   Figure   1.     Tney   include  the   two  most  obvi- 
ously consequential   points  at  which  m i s comm un i c a t ion 
may  occur:    1)    the  reading   and   interpretation  of  the 
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pro-npt,    and   2)    the  reading   and   scoring  of  writing  sam- 
ples. 

Tne    first   point  of  participant  involvement  witn 
texts   in   an   assessment   episode    occurs  when  the  ideas 
and   structure   of  tne   prompt  are   formulated.  Accord- 
ingly,  data  were  collected  during  the   process  of  the 
development  of  a   prompt   for   the   writing  assessment 
under   study.      The   second   point  of  participant  interac- 
tion with  text  occurred  wnen  students  read   the  prompt 
prior   to  writing   their   essays.      In   the   episode  under 
study,    interview  data  ware  collected    from  a  subsample 
of   12'students  before  they  wrote   for   the  assessment. 
During   the  writing  "test"   students  read   the   prompt  and 
wrote  an   essay  in  response.     Data   collected  during 
testing    included   the  written  compositions  of  all  the 
tenth-grade  students  assessed   at  the   high  school,  as 
w-11  as   the  compositions  written  by   the  subsample  of 
students  wno   had  been   interviewed.     The   third   point  of 
participant  interaction   with  texts  occurred  when 
evaluators  read   the  prompt,   read   student  responses,  and 
scored   student  responses.     The  data   collected   at  this 
point  included   the   scores  and   interviews   with  the 
evaluators  conducted   prior   to   the   scoring  of  the  stu- 
d"!nt  essays. 

Evaluation  data   were  collected   from   three  teachers 
who  had   developed   the   prompt   and   an   additional  three 
tea.-hers  at  the  high   school.     These   teachers  partici- 
pated  in   the  holistic   scoring   session.     Another  group 
of  six    teachers  provided   additional  data  by  ranking  the 
papsrs  collected   from  the  subsample  of  students.  In 
addition   to   the  data   collected   at  eacn  point  of 

•XThFFd 'texTT"^*  rubric  or  scoring  guide,  was  not  in- 
cluded in  the  diagram  since  it  was  not  used  in  the 
analysis  reported  here. 
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Tne   tnree   people   involved    in  creating   the  pro-npt 
w^ere  experienced   in    topic  developrnent  and  holistic 
assessment.      The   session  lasted   approximately  2-1/2 
hours,   with  several   pro-npts  being  developed,  con- 
sidered,   and   ultimately  rejecte:i.   In   favor  of  the  one 
under   study.      As   the   pro-npt  evolved,    the   authors  dis- 
cjS3?j   tneir   expectations  of  what  should   and  .-night  be 
fojnd   in   responses  to  various   parts  of  the  pro-npt. 

Tne   session  was   tape-recorded   and   one  :neniber  of 
tne  research  tea-n   attended   the   session.     The  researcher 
tooK  written   notes  on   the  issues  and   the  potential 
p-o-npts  wnich  were  considered,   but  did   not  participate 
in   tne  discussion  except   to  answer  procedural  ques- 
tion s  « 

Twelve  sopnoTjores  were  interviewed   dur-        the  nor- 
mal   writing   assessment  s::heduled   by  the   English  depart- 
ment.     Tne   students   w^re   reconmended   by  their  teachers 
as   average  writers  wno  would   not  be  uncomfortable   in  an 
oral   interview  situation.     One   student  was  later  elim- 
inated  from   the  sample  because  of  problems  in  the 
tape-recording  of  ner   pre-writing   interview.     The  final 
sample   consisted   of  eleven  students. 
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Interview.     The   interviews  were  hel'd  oa  the  iay 
5chedlure~for^tne  writing  assessment  of  the  SDpno-nore 
class.     During   the  period  preceding   tne  s t ud en t ' s  r egu- 
lar   Englisn  .class,   each  of,  the   eleven   t  a  r  g  e  t  s  t  ud  en  t  s 
m^t  one  of  three  researchers   for   an   individual  pre- 
writing  Ir^terview  lasting  ap  pr  ox  i  :n  at  e  1  y  twenty  minutes. 
These   sessiorts  were  tape-recorded.      Researchers  advise^d 
students  that  the   interviews   w,ere  being  conducted^to 
investigate  how  people   react   to   topics  and   plan  their 
CO. u  positions. 

During   the  interview,   stude;it3  w^re  shown   t vfi3 
proTipts:   a   practice  pro.-npt^  and   the  pro-npt  d^sign^d  for 
the  end-of-year   sopno.-nore  class  wr  i  ting  asse'ssment . 
The  practice  prompt  wal^-^used   to  familiarl^ce   the  student 
with  a  "think-aloud"   procedure  and   was  given  l^n  two. 
formats.      In  ohe  format   the   prompt  was  segmenVe^d  into 
phrases.      In  another   format   the  prompt  was  t^i^ped  "in  a 
normal   block  paragraph.     The   practice  prompt  asked  the 
students  to   think  of'a   friend   and   to  describ-  a  situa- 
tion which  revealed   his  or   her  character. 

Daring   each  interview,    the  researcher  presented 
th*  practice  prompt  in   phr a s e -b y-?h r a s e   format,  asking 
the  student  to  react  alouj   to   each  phrase  with  whatever 
ideas  came   to  mind.     Tnis  allowed  eacn  student  to  prac- 
tice? point  by   point  verbalization  of  his  or  her 
interpretation.     The   student   was'  asked   to   continue  to 
"think  aloud"   until   it  was   possible  to   s^ettle  on  a 
specific   content  and   focus   for   a   composition.     Tne  stu- 
dent was   then  shown  the  practice  prompt  typed   in  block 
paragraph  form,   asked   to  re-read   it,   and   instructed  to 
stop  anywhere   in  the  course   of  this  second   reading  to 
verbalize   additional   reactions  to   the  task  presented  l>y 
theprompt. 

After   the  practice  procedure,   the  student  was 
instructed   to   react  to   the  second  prompt,   which'  was 
identified   as   the  one   to  be  used  in   t'he  writing  assess- 
ment.    The  assessment  prompt   was  typed  in  a  normal- 
paragraph   format  as  shown  above.     In  i  t  i  a,l  r  e  s  pon  s  e  s 
were  solicited  by  general  directives  such  as,  "What 
does  this  bring   to  mind?"  Tne   student  was   again  asked 
to   think  aloud   until   the  content  and   focus   for   the  bom- 
'posltion  were  chosen.  * 

Immediately  after  che  interviiw,  tne  eleven  stu- 
dents wrote  compositions  as  part  of  the  regular  assess- 
ment procedure  with  th?ir  classmates.  After  writing, 
they  returned  for  indivi'dual  post-writing  intervie^JS, 
in  which  they  were  asked  about  variations  anong  their 
initial   interpretations  of  the  prompt,   plans  for 
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corn  posing,   and   the  con  positions  that  they  wrote. 


Questionnaire .     On   the  day  following   the  assess- 
ment,   all  of  the  students  (N  s   302)   received   a  ques- 
tionnaire designed  to  find  out   1)  how  the  students  felt 
about  writing  in  general,  and   2)   what  they  thought 
about   the  proTipt.     The   text  of  the  proupt  was  in'cluded 
on  the  questionnaire,   as  was  a  brief  explanation  of  the 
scaie  of  responses.     The   first   18  items  probed  general 
attitudes  toward  writing,  and  elicited   a  co.-npara  ti  ve 
evaluation  of  the  pro:npt  to  others  enco^untered  in 
school.     The   final  nine  ite-ns  asked   the  student  to  rate 
the   importance^ of  satisfying  a  set  of  task  demands 
Which  could   conceivably  be  derived   from  the  statement 
of  the   topic*     A-  copy   of   tne  student   questionnaire  is 
provided   in   Appendix  A. 

filEEIE^  Interpretation   by  the  Holistic  Scorers 

All   of  the  student  coTipo  si  tipn  s  produced   for  the 
assessment  were  scored  on  a  Saturday  morning  one  and  a 
half  weeks   after  tne  day  of  the  assessment.- 

Six   teachers  served   as  holistic   scorers.     Three  of 
the  six   scorers  had  been  involved   in   the  prompt 
development  session,   while  three'were  "naive"   in  the 
sense   that  they  had   not  participated   in   the  formulation 
of  the  prompt. 


Pre-Scqr ing  Interviews .     Interviews  on  the  subject 
of  topic-based  expectations  were  conducted  with  the 
three   naive   scorers  prior  to  the  actual   holistic  scor- 
ing session.     Thf  procedure  for  these  interviews  was  as 
follows:  First,   the  teacher  read   the    assessment  prompt. 
Second,   the   interviewer  asked   the  teacher  to  explain 
what  the  prompt  require^   the  student  to  do.  Tne 
teacher  was   also  requested  to  identify  particular  stra-- 
te^ies  a  student  might  need   to  use  in  order  to  write  a 
good   composition  in  response  to  the  prompt. 

Choosi^ng  Anchor  Papers  >     A  set  of  25  student  com- 
positions was  chosen  at  random  to  be  read   and  scored  by 
all   six  of  the  scorers.     Eacn  teacher  kept  a  list  of 
the  scores  assigned   to  each'paper.     Afte>  reading  the 
set  of  papers,   the  group  tallied  the  scores  received  by 
each  composition. 

The  next  step  in   the  assessment   process  was  to 
select  anchor  papers,   i.e.,   papers^to  exemplify  each  of 
the   five  scoring  levels   fron   1    (lowest)   to  5  (highest). 
The  goal  was  to  find   anchors  which  repre-sented  a  strong 
consensus  in  the  scoring  decisions.     For  example, 
papers  chosen  to  exemplify  a  score  of  ••5"   had  been 
placed   in   that  category  by  all  six   scorers.     In  other 
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cases,   niore  discussion  was  necessary  before  the  te3ch- 
ers  reBChed  a  consensus  on  anchor  papers  to  represent  a 
particular   score.     In   addition,   the  tally  revealed  that 
none  of  this  first  set  of  'papers  nad  earned  a  score  of 
"1",   so  it  was  not  initially  possible  to  identify 
anchor  papers   for   the  lowest  category. 

Development  of  a  Rubric.     Using  the  anchor  papers 
as  guides,   the  teachers  constructed  a  ru:)ric  which 
described   a  prototypical   paper   for   each  of  the  scoring 
levels.     At   each  level,   a  characteristic   paper  was 
described   in  terms  of  content,   f oc us / or g an i z a t i on , 
mechanics,   and  voice/style.     The  descriptions  from  the 
original   version  of  the  rubric  are  summarized   in  Figure 
2. 
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B3C3Use  none  of  the  papers  in   the  initial  scoring 
had  earnai   a  *M  "  ,   tha   t  e  ache  r  s  r  e  v  i*s  e  i   the  rubric  and 
the  ancnoring  scores  to  differentiate  the  two  lowest 
categories  on   the  rubric.     The   revised  rubric  is 
presented   in   Figure  3- 
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The  Scori^£.     Following  established  procedure 
(Myers,    1930),   each  composition  was  read   and   scored  by 
two  people.     A  five-point  scale  based  on  the  rubric  was 
used   to  classify  each  of  the  c  o-npo  s  i  t  i  o  n  s  •     Two  readers 
independently  scored  each  composition.     The   scores  were 
then  combined  to  yield  an  overall   score  for   each  paper. 
Wnen  the  two  scores  showed  a  difference  of  more  than 
one  point,   a  reconciliation  by  a  third  reader  was  used 
to  resolve  the  discrepancy.     The  scoring  session  was 
tape-recorded  . 

The  Ranking  of  th^e  Target  Stajdent  Papers.  To  pro- 
vide additional  data7  a  stcond  d  i  r  5c~e  vaFu  at  i  o  n  of  the 
target  students'  compositions  was  collected.  Six 
experienced  co-nposition  teachers  were  Invited  to  per- 
form and  justify  a  ranking  of  these  eleven  papers.  Tne 
teachers,  who  were  all  high  school  English  instructors, 
were  also  experienced  participants  In  nolistic 
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asssssmsnts   in  their   school  districus.     Z^zn  teaahsr 
was  scheduled   for   an  injiv.idual   session   with  one  of  two 
researchers  on  tne  p'-oject  staff.     Tne  session  included 
a  rankin?  of  the  papers   followed  by  an   interview  in 
which  the  teacners   were  as'<ed    to   discuss   the  criteria 
by  Which  they  assigned   a  rank   for   eacn  paper.  Tne 
interviews   were  conducted  after   the  co.npo  si  t  i  on  s  'had 
been  ranked    in  order   to   avoid   influencing   the  teachers' 
choice  of  criteria. 


The   teachers   were  asked   to  make  written  notes  on 
their   criteria  and   reactions  as  they  went   through  the 
papers.     They  were  able   to  refer   to  these  notes  during 
tnsir  interviews. 
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not 
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a  n 

a 

lysis. 

The   interview  was  designed   to  elicit   the  teachers' 
conception  of  what  constituted   a  well-written  composi- 
tion in  response   to   the   pro.-npt.      It  was  also  aimed  at 
obtaining   the   teachers'    perceptions  of  1)   their  own  cri- 
teria  for  evaluation  of  those  c oupo si t i on s  ,   2)  their 
procedures  for   applying   those  criteria,   and   3)  the 
rationale  benind   their  criteria. 


Ranking   Interview  Procedure .     Tne  ranking  assigned 
to  each  paper  was  recorded.     Tne   interviewer   then  asked 
the  evaluators  to   discuss  their   criteria   for  ranking 
student  papers.     The   following  list  of  questions  was 
not  presented  directly  to  the   teacher,   but  was   used  by 
the  interviewer  as   a  guideline: 

1.  Wnat  do  you  think  a  student  would  have  to  do  in 
order  to  write  a  good  essay  in  response  to  this 
to  pi  c? 


How  auch  weight  did  you  give  to  the 
adherence  to  the  topic/task  demands 
your  criteria   for  evaluation? 


How  much  weight  did  you  give  to  oth 
spelling,    basic  organization,  etc 


student • s 
in  defining 


er   factors   -  e.g 


In   evaluating,   do   you  tend   to   think  of  these 
things  separately  from  the  demands  of  the  topic? 

5.       Wnich  choices   gave  you  the  most  difficulty? 

(i.e.,   choices  between  which  papers?)   and  why? 
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Tne   teachers  were  asked   to   describe  their  criteria, 
procedures,   and   r a t i o n a  1  e ^  f or   ranking   the  compositions 
rfnere  possible,   th3   interv'iewer   attempted   to   follow  th 
teacher's   lead  in  the   discussion^   but  an  effort  was 
uade  to  ensure   that  the  teacher  also  dealt  with  tne 
content  of  all   aspects  of  the  evaluation. 

Several  of  the  rankers   volunteered   insights  or 
opinions  on  writing  instructiont   evaluation  of  writing 
or  related   topics,  wnich  were  developed   through  the  us 
of  non-leading   interview  prompts  whenever  possible. 
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Chapter  4:     Part  II 


THE  WRITING   PROMPT:    DIFFERENCES   IN  EXPECTATIONS 
AMONG  PARTICIPANTS    IN   A  WRITING  TEST  EPISODE 

Charles  Kinzer,   Karen   Carroll,    and    Sandra  Murphy 


Intro d  u  c  1 1 o  n 

In  order   for  a  prompt  to  be  .-neaningful   it  must  be. 
first  of   all.   coniprehenslble  and   second,   capable  of 
triggering  an   appropriate  written  output.     For   this  to 
happen,   certain   aspects  of  Knowledi^e  and  intention  must 
be   coTimun  icated   from  topic   author   to  writer.     A  writer 
can  produce  successful   writing  in  response  to  a  given 
proTipt  when  he  or  sne  can  match  the  prompt  author's 
expectations  of  what  a   "good"   response  will  contain. 
Furthermore,   if  we  measure   success  in  terms  of  grade  or 
score,   the  expectations  of  the  ©valuator  must  be  satis- 
fied by  the  writer's  product.     In  addition,   it  is 
important   to  note  that   evaluators'   expectations  are 
triggered   upon   reading   tne  prompt. 

As   noted   above,   the  study  monitored  a  three-member 
team  during  the  development  of  a  topic  used   to  assess 
ths  writing  of  approximately   300  tenth  grade  students. 
As   they  constructed   the  actual   prompt,   the  teachers 
were  asKed   to   verbalize  any  expectations  they  might 
have  about   the   students'   written  responses   to   the  vari- 
ous parts  of  that  prompt.     The  prompt  wnich  was  eventu- 
ally  developed   appears,   again,  below: 


Many  different  suggestions  for  improvement  of 
Central  High  School   have  recently  been  made. 

scribe  one  problem  or   situation  which  you 
feel   needs  Correction  or  improvement,  giving 
reasons  for  your  choice,   and  suggesting  one 
or  more  solutions* 


The   questionnaire  described  above  was  given  to  all 
participants  in   the  study.     In   addition,  interviews 
were  conducted   with  the  eleven  target   students,  with 
teachers  who  holisticaUy  sctored  the  written  products, 
and  with   five  rankers  who  read  the  topic  and  ranked  the 
eleven  students'   papers   from  best  to  worst . 

The  data  gives  us   insights  into  the  task  interpre- 
tations which  were  "triggered"   by  the   prompt  across  the 
three  sets  of  participants  in  the  assessment  episode: 
prompt  authors,   student  writers  and   holistic  scorers 
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and/or   raters.     Tne   underlying  pre'niS3  of  this  aspect 
of  the  study  is  that  the  nore  closely  the  various  par- 
ticipants in   the  writing  episode  perceive   tne   proupt  as 
being  si-nilar.    the  nore  sjccessful,    in   teriis  of  score, 
the  'response. 

The   Prompt  Development  Session 

Throughout   the   session,   the  prompt  authors  were 
obviously  very  concerned  with  the  implications  of  each 
word   in   the  pro.npt. 

Originally,   the   proupt  asked   for  three  improve- 
ments to   the  school,   out   this   task  was  regarded  as 
being  "too  structured*'.     The   proupt  authors  eventually 
asived   for  a  descriptiua  of  one  problem.      In  their 
words,   they  wanted   to  "force  a  choice"   between  possible 
school   improvenents,     Wnile  students  were  definitely 
expected   to   select  one  problen  or  topic  to  write  on,  it 
was  equally  clear  that  certain  kinds  of  topics  were 
more  "acceptable"   than  others.     The   prompt  authors  were 
somewhat  worried,   for  example,   that  some  responses 
might  expound   upon   the  merits  of  removing  the  existing 
principal   from   the  school. 

Although  not  explicitly  stated   in  the  prompt,  the 
authors  expected   that  the  solution  to  the   problem  would 
be  something  that  "money  can  buy".     At  one  point,  the 
pnrase  "money  is  no  object"   was  written  into  the 
pronpt,   but  this  was   later  deleted.      Although  the 
phrase  was  deleted,   it  appeared   that  the  topic  authors 
expected   that  the  "problem"   selected  could   be  "solved" 
through  an  expenditure  of  funds.     Tney  maintained  this 
expectation  throughout  the  session.     References  to 
monetary  aspects  of  school   problems  and  their  solutions 
are   found   throughout   the  discussion.     For  example,  the 
prompt  authors  felt  that  it  might  be  difficult  to  write 
a  rubric   for  scoring   purposes  that  could  apply  to  tne 
subject  of  physical   Improvements  to   the  school  as  we 11 
to  discussions  of  non-materiaX  changes,   such  as  improv- 
ing  school   spirit.     The  teachers  debated   whether  it 
might  be  difficult  to  equate  material  and  non-material 
subjects.     In   effect,   it  was  evident  that  students  who 
related   their  problem  choices  and  solutions  to  monetary 
factors  would   conform  more  closely  to   the  topic 
authors*    expectations  than  students  who  did  not. 
Nevertheless,    the  authors  concluded   that  the  prompt  was 
"wide  open",   as  long  as  students  could   support  the 
choice  they  made. 

The   prompt  authors  felt  that  it  was  not  enough  for 
tne  writers  simply  to  describe   the  problem.  Responses 
were  expected   to  move  beyond^tne  description  of  a  proD- 
lem   to   include  reasons  why  the  problem  was  thought  to 
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be  a   significant  issuo.     They  also   indicated   that  the 
response  should   show  that  the  problsn  was  related  to 
the   writer,   that  the   solution   would   benefit   tha  writer, 
and   that   the   writer    felt    the    problem    needed  correction. 

Key  words   elicited   specific  expectations  on  the 
part  of  the  topic  authors.     For  example,  "situ?^ion^» 
was   added   to   ••problem"   because   they  felt  that  a  problem 
could   be     cor  r  ec  ted  , but  not  improved.     The  word 
"describe**  was   expected   to   invite  writers  to  begin  with 
strong  detail   and   to  create  a   strong   setting.  '^Many" 
was   expected   to   give  the  reader/writer  an  indication 
that  there  was  no   single,   right  answer   to   the  prompt* 
The   phrase  "at   Central"   replaced   the  original  "ab^the 
school"   because  the  authors  felt  the  latter  excluded 
problems  external   to   the  scnooi  curricuiuii  such  aa 
1  and  sc  api  ng  . 

Tnus  it  appears  tne  topic   authors  assumed  that 
each  word   in   the  prompt   should   have  meaning   for  the 
writer.     Although  certain  parts  of  the  prompt,   such  as 
the   expected   strong   focus  on  one  problem,   appear  to 
have  more  weight   than  others,   each  segment  in  the 
prompt   was   intended   to   provide  a  cue   to   the  writer  for 
what  he  or  she  was  to   put  in   the  response.  More 
research  is  needed   in  this  area  of  possible  within- 
topic   salience.     It   is  probable,  however,   that  most 
prompt   au^nors  would    feel   that   any_  response  must  meet 
the  demands  wnich  are  £5.p_li_ci_t  in  a   prompt,   and   that  a 
"good"   paper   would  demonstrate   compliance  with  implicit 
demands. 

The  H o  1  i  s  t i Scorers 

As  noted   in   PART  I,   six   scorers  holistically  rated 
the  students^    papers.     Tnree  of  the   six   scorers  ware 
involved   in  the   topic  development  session  and  three 
were  '•naive"   in   the  sen'se  that  they  were  not.  Inter- 
views  regarding  topic-based  expectations  were  conducted 
with  the   three   naive   scorers  prior  to   the  group  discus- 
sion which  produced   the  rubric  upon   which  scoring  was 
based.     Naive  scorers  agreed  with  their  counterparts 
who  had   been   involved   in   the   topic's  development  on 
almost  all  points.     Tney  felt  strongly  that  it  was 
important   for   the  students  to   focus   on  one  problem. 
Several   scorers  coiimented  that  the  "structure  of  the 
paper   is  there   in  the  topic/'      Tney  felt  the  main  prob- 
lem  student-writers  would  encounter   was   "narrowing  down 
the   focus.*'      The   following  statements  are  representa- 
tive of  the  naive  scorers'   respons3S  upon   reading  the 
prompt  : 
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"He  (the 
focus  • 
and  give 


writer) 
Then  he 
reasons 


will   have  to  declare  a 

has  to  describe  the  problem 

for  the  c  hoice 


"They're  to  go  from  many  suggestions  to  one 
prob'^lem  and   give  reasons  for   the  choice." 

"If  he  manages  to  avoid   the  list  and  does 
decide  on  one  problem   •    •    •  he'll   have  a  good 
pa  per  ." 

"I  ^ess  the  key  is  finding  that  one  prob- 
lem?^ 

"The   two  pitfalls  may  be  the  list   •   •    •  and 
the  very  general  description  of  the  probiera." 

"It  has  to  be  negative   .    .    .  they  have  to 
persuade  you   .   .    .   they  have  to  argue  (that 
it's  a   problem).     This  calls  for  an  argument 
•    •    •   to  convince  someone   •    •    •   that  it  needs 
correction." 

"The   response  needs  one  problem,  concrete 
examples  and   specifics  beyond  general  state- 
men  ts  . " 


The  above  statements  illustrate  that  the  naive 
scorers'    prompt -based   expectations  are  extremely  simi«- 
lar   to   those  expressed  by  the  prompt   authors.  The 
naive  scorers'    comments  move  beyond   pure   topic  con- 
siderations, however,  and  also   show  a  concern  with 
style,  cohesion  and  tone  or  voice.     The  naive  scorers 
indicated   that  strong  papers  would  be  well-grounded  in 
students*   experiences  at  the   school*-that  better,  more 
credible  papers  would   stay  within   personal  experience. 

Questionnaire  Data  on  Student  and  Author/Rater 
Judgments  o f  Task  Demands 

The   questionnaire   which  students  completed  on  the 
day  following  the  assessment  provided   a  record  of  their 
attitudes  toward  writing,  an  evaluation  of  the  prompt, 
and  an   evaluation  of  task  demands  suggested  by  the 
prompt.     Twelve  of  the   items  on  the   questionnaire  were 
also   presented   to   the  teac he rs  in  the  study.  These 
items  focussed   on  evaluation  of  the   prompt  and  Judg- 
ments of  the  task  demands  it  might  suggest*  Differ- 
ences among  teachers'   and   students'    perceptions  of 
these  demands  are  discussed  below. 

Responses  to   tht  items  on  the  questionnaire  were 
recorded  on  a  scale  of   1  to  5,   with    1  signifying 
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agres'nent  with  a   positive  acatenfient  and  5  signifying 
ajreanent  witn  a   corresponding   negative  stateTient. 
Participants'    responses  are  shown  in   Table    1  below. 

Percentage  of  Responss  by  Students  and  Proupt 
Authors/Raters  to  Selected  due  st  i  onna  J  ri^  Ite-ns* 

igree      Neutral  Agrei* 


A.  1,     The  topic  Is  more 
p 1 e Ing   to  wr  i  te 
about  than  nos t 
others  given  In 
school • 


Thp;   topic  Is 
to  read  and 
understand. 


easy 


31 

70 


7B 

80 


37 

20 


18 

20 


31  The  topic  Is  less 
10    pleasing  to  write 

about  than  most 

others  given  in 

s  ch  ool • 

3  The  topic  Is  difficult 
0     to  read  and  understands 


3. 


The  topic  Is 
atr  i  te  about  • 


tasy  to 


B 


9. 


ID. 


II. 


12, 


I  think  the  topic 
1^11  1  gI vc  a  good 
f  nrl  I  cflit  T  on  of  my 
i the  student's  ) 
wr  J  t Ing  ab  i 11 1  y • 

It   Is   I-nportant  to 
des cr  Ibe  onl y  one 
p  robl e«» 


60 
SO 

21 

70 


35 
90 


It  Is  iTiportant  to  B2 
to  expl  al  n  why  the  90 
subject    Is  a  problem. 

It   Is   I"nportant  to  B4 
sujjest  a   solutlDn  SD 
to  the  probleti  or 
s  » tua  1 1  on . 

It   IS  i-nportant  to  b9 
su^jest  more  than  19 
one   solution  to  the 
p  r  1  b  1  e  -n . 

1 1   Fs   Important  to  53 
point  out  that  many  0 
suggestions  for 
improvinn   the  school 
have  been  m.ide  • 

It   Is  iTiportant  to  71 
relate  the  problen  io 
to  everyone  In  the 
school • 

It  Is  Important  to  51 
show  how  the  problem  ZQ 
affects  the  wr  Iter, 

It   Is   Important  to  77 
explain  how  the 
solution  will 
Improve  the  school  . 


23 

20 

37 

20 


0 


13 
10 


1:^ 
20 


22 

20 


38 

10 


22 


36 
20 


19 
^0 


17  The  topic  fs  difficult 
2w     to  write  about • 

^2  I  th  ink  the  topic  will 
10    not   give  a  good 

Ind I  cation  of  my 

(the  student's) 

wr I t  i  ng  ability. 

21     It   Is  not  Important 
10     to  describe  only  one 
p  rob  1  efB# 

3     It   Is  not   Important  to 
0    explain  why  the  subject 
Is  a  problem* 

2     It   Is  not   Important  to 
D     suggestion  a  solution 

to  the  problcfl  or 

s 1 1 ua  t  Ion • 

7    I t   I s  not  important  to 
70     suggest  jiore  than  one 
sol  ut  Ion  to  the 
Probl em» 

9     It   Is  not   Important  to 
90     point  out  that  iiany 
suggestions  for 
Imorovina  the  school 
have  been  made* 

7    It   Is  not  Important 
80     to  relate   the  problem 
to  everyone   In  the 
school • 

12     It   Is  not    Important  to 
60    show  how  the  problem 
affects  the  wr Iter • 

2     It   Is  not  Important 
20     to  explain  how  the 
sol  ut  Ion  win 
Improve  the  school  • 


•The  top  row  of  numbers   In  each  case  refers  to  students  (W.302S 
?nr^^w  1°^.^°  ^""^^  -uthors/raters  CN-IO).     SoT^e  rows  do  not  toil)' 
100«  due  to  rounding  error  and/or  missing  responsss. 
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Tne  topic 
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is  easy  to 

write  about. 
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e  topic  wi 
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c  a 
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)   w-iting  ability. 

It   is  interesting  that  students  found   the   topic  far 
less   pleasing,    compared   to   other   topics  given  in 
school,   than   the  other   participants.      Also,    there   is  a 
definite  disagreement  between   the  students  and  teachers 
as  to  whetner  or   not   the  writing  produced   by  the  prompt 
would   give  a   good   indication  of  writing  ability,  Tnis 
disagreement  snould   be  cause   for   concern,  especially 
since  students   indicated   on  other   portions  of  the  ques- 
tionnaire  that   writing  is   an   important  tning   to  learn 
(34X)   and   that  ability  to   write  well   would   be  important 
after  leaving  school  (901). 

Tne  combined   group  of  evaluators,  therefore, 
appeared   to  have  a  different   idea  of  the  relevance  or 
quality  of   the   prompt   than  did    the   student  writers. 
One   ranker  did   recognize   this.      She   felt  that  the 
prompt  would  not  give  a  good   indication  of  writing 
ability,   because,   as   she   argued,   topics  n^eed   to  be 
relevant  to   the  writer  and   the  situations  discussed 
must  be  perceived  as  being  within   the  writer's  control. 
Sne   pointed  out   that  students  Tiay  be   tired  of  writing 
aoout   school   problems  wnich  they  cannot  directly  influ- 
ence and   which  must  be  solved   by  adults.      In   fact,  she 
felt   th3t   students  might   resent  having   to   spend  their 
time  providing   solutions  which  no  one  would  implement 
and   that  students  .might  recognize   that  no  one  with 
decision-making   power   would   read   their  suggestions. 
One  of  our   subsample  responses   illustrates  her  point. 
The   student  recognized   that  sne  was  powerless  to  pro- 
vide a  solution,  saying: 
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"The  only  »^ay         can  solve  this  protDle.Ti 
is. ..have  our   parents  speak  for  us,  Tnay 
(school   board)    should   listen   to   the-n  becaus: 
they  are   the  ones  who  pay  taxes.*' 


Percentages  of  responses  by  the  two  groups  to 
statements  about   tne   importance  of  including  certain 
elements  in  compositions  written  in  response  to  the 
prompt  can  be  seen   in  Part  B  of  Table  1. 

Students  and   teacners  appeared   to   agree  that  it 
was   important  to  explain  why  the  subject  is  a  problem 
and   to   suggest  a   solution*     More  than  80S  of  both 
groups   agreed   to  both  of  the   following  statements: 


It  is  important  to  explain  why  the  subject  is 
a  problem. 

It  is   important  to   suggest  a   solution   to  the 
problem  or  situation. 


Among   the  many  differences   found   between   teachers  and 
students,  one  of  the  most  dramatic  was  their  opposing 
views   on   the   importance  of  mentioning   tne  suggestions 
wnich  had  been  deoated-in  the  months  preceding  the 
assessment.      Fifty-three   percent   (53X)   of   the  students, 
but  none  of  the   teachers,   agreed   with  the  following 
statemen  t : 


It   is  important  to   point  out   that  many 
suggestions   for   improving   Central   High  School 
have  been  made. 


In  other  words,   a  majority  of  the  students  thought  they 
Should   reference   those   suggestions,   while   none  of  the 
teachers  thought  a   listing  of  those  suggestions  was 
important. 

Another  difference   across   participant  groups  can 
be  noted   in   responses  to   the  statement: 


It  is  important  to  describe  only  one  problem. 


In   the  topic  development  session   tne   prompt  authors  had 
indicated   they  felt  this  was  a  vital   element  in  the 
prompt.     At  one  point  in   the   topic  development  session, 
the   following  exchange  occurred; 
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a:       '»Do   W2   like   iiniting  it  to  one  proDlen? 
Tnat's  what  we  will   loo<  for.** 

b  :       "  I  d  o  . 

c:       "I  do,  roo.^» 

In   tne  questionnaire  data  this   stress  on   the  importance 
of  one   topic   in   the  responses  is  confir.Tjed   by   the  coti- 
bined  group  of  evaluators.      Ninety  percent  of  the 
evaluators  felt  a   focus   on  one   problem  would   be  a  vital 
part  of  a   response   to   this   prompt.     Yet   these  expecta- 
tions were  not  matched   by  the  student  writers.  Only 
35X  of  the   students  agreed   that  it  was   important  to 
describe   only  one  problem. 

The  disagreement  on   the  nunber  of  problems  to 
include  in   the  composition  was  consistent  with  the 
disagreement  on   the  number  of  solutions  to  suggest. 
Sixty-nine   percent   (69X)   of  the   students  agreed  with 
the   statement:   "It   is   important  to  suggest  more  than 
one  solution."   while  only   10%  of  the   teachers  thought 
this  was   necessary.      However,   this  result  may  also 
reflect   some   students'    responses  to   the   prompt  segment 
".suggesting  one  or  more  solutions." 

Tne   two  groups  of  participants  also  appeared  to 
disagree  about   stressing  other   elements  in   the  composi- 
tions.    Many  of  the  students  emphasized   elements  which 
were  not  .considered   important  by  the   teachers.  For 
example.    51X  of  the  students  agreed  with  the  statement: 
"It   is  important  to   relate   the  problem   to   everyone  in 
the  school."   but  only   lOt  of  the   teachers  did. 

Differences  Between  Teacher  and  Student 
Tn  t e r p fe^ a  t i o n s^oT ""T a s k  Demands 

The   examples   from   the  interviews   and   the  question- 
naire data   indicate  that  teachers  and   students  often 
perceived   task  demands  differently.     One  major  differ- 
ence seemed   to   be  that  the   prompt  authors  lent  weight 
to   all   parts  of  the  prompt:   Tney  felt   that  every  word 
wasTmportant .   both   for   the  explicit  meaning  and  the 
implicit   requirements  it  conveyed.     Some  of  the 
student-writers,   on   the  other   hand,   focussed   on  only  a 
few  selected   aspects  of  the  prompt. 

Ip   the   prompt  development   session   the  prompt 
authors   appeared   to   feel   that  the  first   sentence  pro- 
vided  adequate  but   non-prescriptive  c o n t e x t u a  1 i za t i o n  . 
allowing   students   to   proceed   to   their    selection  of  3 
specific   problem.     However,   only  one   student  immedi- 
ately recognized   that   the   first   sentence  was  intended 
to   function  merely  as   an   introduction   to   the  prompt. 
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rather  than  as  an  integral  part  of  the  topic.  Tnis 
stujent  said: 


The   way   I  interpreted   that   (the   first  sen- 
tence)  was   just   as  an   introduction    ...  I 
figured  they  ware  just  s  ying  this  so  they 
could   tell  you  to  describe  one  problem   .    .  . 
I  didn't   think  that  was   important   .    .    .    .  Thtf 
most  important  part  was   (the   begn^nning  of  the 
second  sentence). 


Another   student  d eno n s t r a t ed   that  she  was  aware  of 
two  possible  ways  of  treating  the   first  sentence.  Ini- 
tially,  she  failed   to   find   the  task  interpretation 
favored  by  the  teachers  who  developed   the  prompt,  i.e. 
a  focus  on  a  single  problem.     She   initially  stated: 


First   I'd  talk  about  all   the  suggestions  that 
were  made  to   improve  Central,  and   then  narrow 
it  down  to  the  best  few  and   then   (to)  the' 
main  thing  that  neeis   improving.  (Carla) 


Subsequently,   in   hsr   post-writing   interview  this 
same  student  indicated!   that  she   rejected  her  initial 
interpretation  before  writing: 


I  was  thinking  of  doing  that  but  then  I  read 
the  topic  again  ...  I  Just  thought  It'd  be 
best   to   start  right  In   (on   one  problem). 

An  examination  of  the   prompt  used   in  this  study 
reveals  that  explicit  directives   include  the   focus  on 
only  one  problem,   which  the  student  feels  needs  to  be 
solved,   and   the  Inclusion  of  reasons  and  one  or  more 
solutions.     However,  some  of  the   teachers'  expectations 
are  not  explicitly  stated   In  the  prompt.     For  example, 
(In   the  prompt  development  session)   ••to   tell   why  It's  a 
problem  or   situation"   was   the  I n tend ed  mean  I ng  for 
"giving  reasons  for   your  choice.'*   Beyond   that,  many  of 
the   students  felt  It  was   Important  to  mention  that  many 
"suggestions"   had   been  made,  while  none  of  the  teachers 
felt  this  was   Important.      It  appears,   therefore,  that 
many  of  the  students  were   Interpreting  the  task  sug- 
gested by  the  prompt   In   a  different  way  than   the  prompt 
developers  Intended. 

Tne   prompt  or   topic   statement  Is  the  linking  ele- 
ment across  the  three  segments  of  a  writing  episode. 
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Tnerefore,   It  Is  Injportant  that  the  students,   the  test 
makers  and   test  scorers  agree  on  what  the  prompt  is 
asKing   the  student  to  do.     Otherwise  the  scoring  rut)ric 
may  not  reflect  the  criteria  the  prompt  writer  origi- 
nally had   In  mind,   and  scoring  the  student  papers  may 
be  furtner  complicated  if  the  students  view  the  tasiC 
presented  by  the  prompt  quite  differently  than  the  test 
ma;<ers  Intended. 

Questionnaire  Re^pon ses  ajnd  Total  Holistic  Score 

Kendall's  coefficient  of  correlation  was  computed 
for  total  holistic   score  and   students'    responses  to  the 
pairs  of  items  in  Table   1   above.     Complete   texts  of  the 
student  and    teacher   questionnaires  are   included  in 
Appendix   A,    and  Q. 

The  measure  of  association  was  computed  for  botn 
the  target  student  group   (nrll)   and   the  larger  group  of 
students   (n=302).     Results  are  shown  in  Table  2* 
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Table  2 


Kendall's  CorrelatTon  Coefffclents  of  Respcn 
Selected  Quest  Icnna i r e  Items  With  Total  Holfst 


1.  The  topic  Is  frcre 
pleasing  to       i te 
about  than  most 
others  given  In 
school • 

2.  The  topic   IS  easy 
to  read  and 
understand* 


3. 


1. 


The  topic  Is 
write  about. 


easy  to 


I   think  the  topic 
will  s  I  ve  a  good 
I nd  i  cat  I  on  of  my 
Cthe  student's) 
wr  i  t  i  ng  ab 1 1  I ty • 

It  Is  Important  to 
descr  Ibe  cnl y  one 
probl em. 

It  Is  Important  to 
to  expla in  why  the 
subject   fs  a  prcblerr* 

It  1$  Important  to 
suggest  a  solution 
to  the  problem  or 
s I tua t Ion  • 


.07 

'.196 

-.02 
•  01 

-.02 
-.16 
-.13 


E 

.035 


.001  = 


.297 


.476 


.289 


.001* 


.001* 


e.     It   Is  Important  to  ,03  .238 

sugge 5t  more  than 
one  solution  to  the 
problem. 

9.     It   ts  Important  to  .02  .321 

point  out  that  many 
suggestlrns  for 
Improving  the  school 
have  been  made. 

10.  It   Is  Important  to  •,09  .012 
relate  the  problem 

to  everyone  In  the 

school m 

11.  It   Is  Important  to  -.02  .243 
show  how  the  problem 

affects  the  writer. 

12.  It   Is  Important  to  -,11  ,002^ 
explain  hew  the 

solut  ion  will 
'Improve  the   school,  n-302 


ses  to 
ic  Score 


-.06  .411 


*.09  .340 


-.05  .410 


-.12  .303 


.43 


-.29 


-.28 


.05 


-.25 


-.27 


-.59 


n-11 


.032 


.105 


•  118 


.417 


.417 


•  146 


•  123 


•  006 


«*1pha  «  .05/1? 
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In  the  larger  group  of  students  surveyed,  signifi- 
cant associations  were  found  between  total  holistic 
score  (i.e.  the  combined  scores  of  two  evaluators)  and 
four  of  the  Items.     The  results  indicate  that  students 
who  rated   the  prompt  as  easy  to  read  and  understand 
tended   to  receive  higher  holistic   scores.     High  scores 
were  also  associated  with  agreement  that  it  was  impor- 
tant to  explain  why  the  subject  chosen  was  a  problem, 
and  that   it  was  important  to  suggest  a  solution  to  the 
problem.     The  results  also  indicate  that  students  who 
agreed  with  the  statement  that  it  was  important  to 
explain  how  the  sol ut ion/ ^^r f ec t ion  of  the  subject 
chosen  would   improve  Central  High  tended   to  receive 
higher  scores.     Significant  associations  were  not  found 
for   responses  to  any  of  the  other  Items. 

In  the  target  group  of  eleven  students,  signifi- 
cant associations  between  questionnaire  responses  and 
holistic   score  were  not  found. 


Discussion 


It  is  important  to  note  in  discussing  these 
results,  that  although  some  of  the  measures  of  associa- 
tion are  significant,  they  are  very  low.     However,  if 
one  considers  that  many  factors  influence  the  evalua- 
tive Judgments  teachers  make  in  reacting  to  a  written 
composition,   including   factors  which  are  not  related  to 
features  of  the  prompt  such  as  legibility,  errors^ 
spelling,  etc.,  the  low  correlations  should  not  be 
surprising.     Given   this  qualification,  the  results  do 
support  the  viewpoint  that  there  is  an  important  con- 
nection between  reading  and  writing  in  a  writing  test 
situation  (Ruth,  1982). 

In  his  review  of  the  literature  on  topic  effects, 
Hoetker  (1981)  suggests  that'  read  ab  il  it  y  and  students' 
reading  skills  are  factors  that  should  be  considered  in 
prompt  design  and  test  evaluation.     These  results  sup- 
port the  notion  that  students  with  "reading"  problems 
are  also  likely  to  have  difficulty  in  making  an 
appropriate  response  in  a  writing  assessment.     In  the 
present  study,  students  who  found  the  prompt  ••easy  to 
read**   received  higher  scores. 

Hoetker  has  also  pointed  out  that  students  might 
have  difficulty  reading  ••between  the  lines^*   to  inter- 
pret the  implicit  task  demands  suggested  by  a  prompt. 
In   this  context,  it   is  important  to  point  out  that  two 
task  demands  which  were  not  stated   explicitly  in  the 
prompt  were  significantly  correlated   with  holistic 
score.     Agreement  with  the  statements  ••It   Is  important 
to  explain  why  the  subject  chosen  is  a  problem  at  Cen- 
tral," and   ••It  is  important  to  explain  how  the 
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solution/correction  of  the  subject  chosen  will  mprove 
Central  High,"  were  significantly  correlated  with 
holistic  score.     Yet  explaining  why  the  subject  Is  a 
problem  Is  not  precisely  equivalent  to  either  "Describe 
one  problem."  or  to  "giving  reasons  for  your  choice. 
Similarly,   explaining  how  the  subject  chosen  will 
Improve  Central  High,  Is  not  explicitly  stated  as  a 
demand  of  the  task.     Students  who  consider  these  tasks 
unnecessary  elaborations,  and/or  students  who  interpret 
the  task  differently  may  be  evaluated  in  part  for  their 
interpretation  of  the  prompt.     Students  who  agreed  that 
these  task  demands  were  Important  were  awarded  higher 
scores . 

One  would  expect  that  task  demands  that  are  stated 
explicitly  In  the  prompt  would  be  correlated  with 
score.     Thus  it  Is  not  surprising  that  students  who 
agreed  with  the  statement  "It  is  Important  to  suggest  a 
solution  to  the  problem  or   situation"  received  higher 
scores.     However,  the  fact  that  higher  scores  were 
awarded  to   students  who  recognized  the  importance  of 
unstated  task  demands  suggests  that  students  whose  task 
"interpretation"  matched  the  intention  of  the  evalua- 
tors  in  regard  to  these  unstated  elaborations  were  more 
likely  to  satisfy  their  expectations,  and  perform  well 
In  the  assessment. 

The  question  of  whether  or  not  students  actually 
interpret  the  prompt  In  different  waVs  Is  addressed 
more  fully  In   Part  II  of  this  report/.     One  should  note, 
however,  that  various  Inter pretatlonb  of  the  phrase 
"giving  reasons  for   your  choice,"  ar^  possible.  Includ- 
ing giving  reasons  for  why  the  problek  should  be  con- 
sidered a  problem,  giving  reasons  for  why  the  problem 
was  chosen  above  others,  and/or  giving  reasons  for  why 
the  problem  was  personally  relevant.     Given  that  higher 
scores  were  awarded   to  students  who  agreed  that  it  was 
Important  to  explain  why  the  subject  chosen  was  a  prob- 
lem, it  appears  that  the  first  interpretation  may  have 
been  favored  by  the  evaluators.     A  recognition  of  the 
Importance  of  including  this  explanation  requires  the 
student  to  go  beyond  the  explicitly  stated  requirements 
of  the  prompt  in  order  to  match  the  evaluators' expecta- 
tions . 

The  data  presented   here  clearly  indicate  that  the 
expectations  of  evaluators  differ  from  those  of  stu- 
dents with  regard  to  the  prompt  used  in  this  assess- 
ment, and  that  implicit  task  demands  may  have  an  effect 
on  the  way  students  are  evaluated. 
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Chap  t  er   4  ;      Part  III 


THE  WRITING  PROMPT:  THE  PROCESS  OF  INTERPRETATION 
AND   PERFORMANCE  THAT    IT   ELICITS    IN   STUDENT  WRITERS 

Sandra   Murphy,   Karen   Carroll,    and  Ann  Robyns 


Assumptions  Underlying  the  Analysis 

Part   II  of  this  report  has  dealt  with  differences 
anong  the  prompt-based  expectations  of  groups  of  lndi<- 
vlduals  with  regard  to  performance  and  Interpretation 
of  task.     In  the  present  discussion,  we  will  be  focus- 
ing our  attention  on  the  wording  of  the  writing  prompt, 
the  Initial   (oral)   in ter p.^e^ations  of  the  prcn:pt  by  the 
subsample  of  students  described  above,  the  ways  these 
students  Interpreted   the  prompt,  and   the  ways  the  Meas 
they  generated  were  translated  Into  written  prose.  The 
data  we  will  be  dealing  with  here  Include  the  oral 
pre-wrltlng  Interviews  and  the  written  compositions 
collected   from  the  sub-sample  of  eleven  students  during 
the  writing  assessment  episode  and  their  post-writing 
In  terv  lews  • 

An  underlying  assumption  In  our  analysis  was  that 
the  student  accesses  Information  In  the  process  of 
reading  the  prompt  which  he  or  she  then  uses  In  compos- 
ing a  written  response.     The  construction  of  an 
Interpretation  of  the   prompt  Is  thus  viewed  as  an 
Integral  and   Important  part  of  the  composing  process. 
In  the  process  of  Interpreting  the  prompt,  the  student 
■ay  selectively  focus  on  some  segments.  Ignore  others, 
and   Interpret  the  ones  assimilated  differently  than  do 
the  evaluators  In  the  assessment.     Thus,  the  student 
may  "construct"  an  Interpretation  of  the  prompt  which 
differs  from  the  Interpretation  of  other  participants 
In  the  assessment. 

While  Part  II  of  this  study  revealed   that  students 
arrived  at  conclusions  about  task  demands  which  dif- 
fered  from  those  of  the  prompt  developers  and/or 
evaluators,  the  present^ analysis  deals  with  variations 
In  the  ways  the  eleven  students  constructed  interpreta- 
tions of  the  prompt. 

Analysis  of  the  Prompt 

Our  analysis  Includes  an  examination  of  the  form 
and  content  of  the  written  prompt,  and  examinations  of 
the   form  and  content  of  students*   oral   responses  and 
written  compositions.     Each  of  these  three  sources  of 
information  was  considered   first  in  isolation,  and  then 
as  an  interactive  element  in  a  single  discourse  text. 
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Each  source  of  data  served   to  shed  light  on  the 
others*     At  times,  student  responses  focused  attention 
on  problematic   features  within  the  prompt.  Conversely, 
certain   features  of  the  prompt,   which  were  analyzed  as 
being   ambiguous,  suggested  where  we  might  look  for 
potential   areas  of  confusion  within   student  responses. 
By  considering   the  interaction  of  the   three  sources  of 
data    1)   the  prompt,   2)   the  initial 

interpretation/response,   and   3)   the  written  composi- 
tion,  we  were  able  to   identify  various  interpretive 
strategies  used   by  the  students  in  responding  to  the 
structure  and  content  of  the  prompt  within   the  context 
of  this  assessment  situation. 


Cohesion  and  Contextual ization 

Haviland  and   Clark  (197^,   p.   3)   have  defined  given 
information  as:     '•information  the  speaker  considers 
given  —  information  he  believes  the  listener  already 
knows   and   accepts  as  true.**  They  define  new  information 
as:   '•information  the  speaker  considers  new--informat  ion 
he  believes  the  listener  does  not  yet   know."   The  terms 
given   and/or  "old"  as  they  are  used  here  refer  to 
knowledge  that   is  shared  between  "speaker"   and  "hearer" 
(in   this  case,   prompt  developers  and   student  writers). 
When  considered   as  an   isolated   text,   there  are  two  ways 
in  which  the   following   sentence  could  be  read: 

Many  suggestions  for   improvement  of  Central 
High  School  have  recently  been  made.  (SI) 


If  the  reader   knows  about   "suggestions  recently 
made,"   SI  may  function   as  an   anchor-point  reference  to 
the  context  of  a   familiar  discussion.     In  contrast,  if 
SI  is   treated   as  "new"   information,  being  introduced 
for  the   first  time,  it  will   inform  the  reader  of  an 
existing   state  of  affairs  of  which  he   is  as  yet 
una  war  e  • 


If  the  reader  treats  the  existence  of  "many  dif- 
ferent suggestions  recently  made"  as   "new"  information, 
it  is  likely  that  the  reader  will   expect  unresolved 
elements  of  this  piece  of  news   to  be  qualified.  Ordi* 
narily,   new  information  is  qualified,   illustrated,  or 
expanded   in   the  sentences  that  follow,  to  ensure  that 
the  reader   is  sufficiently  anchored   in  the  intended 
subject  context.     This  expectation  is  related  to  the 
process  of  finding  coherence  between  one  sentence  and 
the  next. 


Fillmore  ex pr esses  the  relationship  bet ween  coher- 
ence  and   co n t e x tual i za t i o n  In   the  following  statement: 
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A  text  can   be  said   to  be  coherent  if  a  single 
i.uagined  world   is  conpatible  with  all  parts 
of  the  text,   (  1  975a,  p.  137) 

Wnen   the  reader  can   find  no  qjalification,  illustration 
or  expansion  of  Si  by  S2,   he  cannot  readily  interpret 
the  two  sentences  as  being  coherent,   and  may  be  forced 
to  re-exa:nine  his  initial   interpretation  of  the  first 
sentence . 

Research  in   the  fields  of  cognitive  processing  and 
reading  comprehension   supports   the  view  that  readers 
expect  the   information   In  adjacent  sentences   to  be 
related,   and  that  it  is  easier   for  readers  to   form  an 
integrated  representation  of  related   pieces  of  informa- 
tion if  the  explicit  markers  of  cohesion  in  adjacent 
sentences  are  provided   (Bransford  and   Johnson,  1973; 
Goldstein,    1930;  Halliday  and   Hasan,    1976;  Haviland  and 
Clark,    197U;  Hayes-Roth  and  Thorndyke,    1979;  Klntsch, 
1975;  Moe,    1979;   Walker  and  Meyer,    1930;  Yekovich  and 
Walker,    1978).     Tne  lack  of  direct  links  between  the 
two  sentences  in   this  prompt   was    viewed  as   a  potential 
source  of  difficulty  for  the  students  in   the  assess- 
men  t  • 

Coherence  and  Un  i  f ylng  Purpose 

Another  criterion   specified  by  Fillmore   for  coher- 
ence within  a   text  involves  "a  more  or  less  unified 
answer   to  qjestions  about  the  speaker's  or  author's 
purpose,"   (1975a,  p.    137).     If  the  reader  does  not 
treat  the  information   in   the   first  sentence  as  new.  If 
he  treats  it  instead   as  given   Information,   or  informa- 
tion which  refers  to  shared  knowledge,  he  Is  forced  to 
qjestion   the  purpose  the  sentence  would  serve.  The 
reader  might  ?sk  ,^for   example:   ••Is  this  the  subject  of 
the  composition   I  am  about  to   write,   or  is  It  intended 
to  be  a  background   for  what  follows?"   In  either  case, 
whether   the  reader  treats  the   first  reference  to 
"suggestions"   as  a  reference   to  old  or  to  new  informa- 
tion, he  Is  forced  to  search   for  a  context  and/or  a 
unifying  purpose  with  which  to  relate  the  two  parts  of 
tne  text  to  one  another  because  explicit  marks  of  cohe- 
sion between  SI  and   S2  do  not  exist. 

In   order   to  arrive  at  a  single  coherent  interpre- 
tation,  the  reader  must  read   selectively,  applying 
abductive  reasoning  to  the  task.     This  process  requires 
a  certain  degree  of  subtlety  on   the  part  of  the 
reader--to  know  how  and  when   In  manipulate  the  pieces 
of  the   text  to   compensate   for   an   inherent  lack  of 
coherence  resulting  in   ambiguity.     This  means  knowing 
how  and   when  to  manipulate  whatever  schemas,   shared  or 
personal   scripts,  assumptions  or   facts  the  reader  has 
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available  to  him,  to  second  gu-2ss  the  author's  intenjeJ 
mean  i  n  g  . 


rfe  have,   thus  far,   dis2uss2d   the  lack  of  explicit 
marks   of  cohesion  between  Si   and   S2  of  the   prompt.  rfe 
have  argued   that  this  constitutes  an  inherent  source  of 
ambiguity,   resulting  in   two  possible  interpretations  of 
the   full   text.     In   the  following  we  will   examine  sen- 
tence  internal   presuppositions  and   cross-reference  sen- 
tence  internal    information  with  sentence  external 
information   to  determine  how  each  may  influence  the 
other   in   the  construction  of  two  possible  interpreta- 
tions . 


Text/Semantic  Analysis 

The  concept  of  presupposition  as  originated  in 
logical   theory  appears  in  variously  amended   forms  as 
applied  by  linguists  in   semantic  analysis  (Ruth, 
1930,   p.    129).     For  example,   it  is   logically  possible 
to  distinguish  between  what  a   speaker  asserts  and  what 
he  presupposes,   i.e., between   what  he  says   and  what  he 
takes   for   granted   as  being  true.     The  logical  presuppo- 
sitions of  a   declarative  sentence  are   those  proposi- 
tions that  can  be   inferred   in   isolation   from  any  con- 
text and   that  must  be  true   if  tne  assertion  as  a  whole 
is  true.     Fillmore   (1971)   expands  the  concept  of 
presupposition  to   include  the   speech  communication 
situation,   wnich  he  analyzes   in  terms  of  two  levels: 
the   presupposi  tional  ,  and   the   i  1 1  ocu  t  i  o  nar  y  .  •••In 
every  conversation,   we  constantly  make  use  of  both  the 
Implicit,  or   presupposi t lonal ,   and   the  explicit,  or 
illocutionar  y  levels  of  commun  ica  tion  ,  •  »•  (Fillmore. 
1971i   p.   277),     ThuSt   the  construction  of  the  meanings 
of  given  sentences  necessarily  requires  building  a 
structure  comprised  of  explicit  referential  information 
plus    implicit  presupposed   information.     At   this  point, 
we  should   note  that  most  theoretical  discussions  of 
presuppositional   analysis  pertain  to   Ideal ized  language 
users.     In  this  case,  we  will  apply  our  analysis  to  a 
real   text,   the  Central  High  School   writing  assessment 
prompt,  but  we  will  describe  the  several  idealized 
paths  of  interpretation  readers  might   take  through  the 
text  of  the  prompt.     Our  aim  is  to  display  several  dif- 
ferent potential  constructions  of  meaning  which  all 
make   sense  and  may  account  for  variations  in  comprehen- 
sion of  the  prompt.     The  text  of  the  prompt' is  repeated 
verbatim  here   for  reference  throughout   the  discussion 
below. 


Many  different  suggestions  for   improvement  of 
Central   High  School   have  recently  been  made. 
Describe  one  problem  or   situation  wiich  you 
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feel  needs  corre 
reasons  for  you^ 
or  more  solution 


c  t i  on  or 
choice, 
s  . 


i-nprove-nent,  giving 
and   sjggestin?  one 


In   Sentence   1    (31),   the   passive  verb  construction 
may  shift  the   reader's   focus  of  attention   from  the 
natural   (cognitional*)    subject   (the  presjpposed  source 
of  "suggestions")   to   the  natural   (cognitional)  object 
(the   result  of  the  verb   process,   "suggestions"),  here 
appearing   in   subject  position.      If  SI   had   been  cast  as 
an   active  verb  construction,   the  terra  "suggestions" 
would   have  appeared   in   its  natural  order   in  object 
position.     Thus,   there  would   have  been  represented  an 
actor-action-goal   "picture  of  reality"   at   Central  High 
School   instead  of  the  reversed   relation  of  goal- 
action  and  descriptive  qualifiers. 

'rfnen   the   plural  marker  "s"   and  qualifiers  "many" 
and  "different"   are  associated   with  "suggestion,"  the 
connection  makes   several   plausible  constructions  of 
meaning  or   paths  of  interpretation  possible.     The  fol- 
lov^ing   possibilities  arise: 

1.  many  different  sources  of  opinion  are  engaged  in 
tne  one  process, 

2.  one  source  of  opinion  is   engaged   in  many  different 
pro  cesses  , 

3.  many  different  sources  are  engaged   in  many  dif- 
ferent processes. 


Prior   experience  with  "suggestion -making"  situa- 
tions may  guide  the  reader  toward  a  pro  to  typical 
"scenario".     For  example,  one   such  scenario  might 
involve  multiple  sources  of  opinion  (representatives), 
offering  many  different   suggestions  in   a  group  context 
(committee)   on  one  or  a   few  occasions  (meetings). 

Further   information   is  available  to   the  reader 
about   the  e v en t-pr o c e ss ( es )   in   the   form  of  an  adverbial 
modifier   "recently"   which  circumscribes  the  event- 
process(es).      In  combination  with  the   past  tense 
marker,   the  modifier   effectively  completes  or  finishes 
the  e V  en t-pr oc e ss ( es )  ,   and   places  them  in   the  "recent" 
past.     "Suggestions  have  been  made"  orients  the  reader 
to  a  past  event  and   "recently"   places   the   event  in  the 

term  "^in  trod  uced"^by"l?3nTr  ed  Sandmann  (  1  95  ^  )  to  con- 
trast with  "grammatical"  subject  to  avoid  the  confu- 
sions of  "logical"  and  "psychological"  which  have  been 
applied   variously  by  language   scholars   (p.  237). 
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recent  past,  makini  the  event  a  precedent  for  w'nat  ttia 
stiudant  writsr  is  about   to  un  d  er  ta /ce  . 


In   Santence  2,    tha  ^nodifler   '•one/'    is   pos2d   as  a 
qjantitative  constraint  on   tha   njnber  of  "probleTis  or 
sitjations"    to   be   described.      It  can   also  be  inter- 
preted,   especially   »^,4en   SI   is   acting   as  old  inforTig- 
tion,   as   an   indefinite  pronoua^.      Based   on   the  presuppo- 
sition  that  many  different  opinions  already   exist   as  to 
wnat  constitutes  a   "probleu*'   or   problematic  "situa- 
tion/'   wnat  constitutes   a   "needed   correction^'  or 

impr  ov  em  en  t/'    and    finally,   what  constitutes   a  valid 
"suggestion,^    the   reader  may  conclude   that   he  is 
required    to    reference  "£  suggestion,"    and  describe 
"one"   of  tnese,   i.e.,   "one"   fron  a  defined   set  of  "many 
different  suggestions/ 

The   progress  of  the   reader   is   further  complicated 
by  the   presence  of  imparfective   processes   in   both  SI 
and   S2.      Tnese   inherently  partial,   or  incomplete 
processes   are  signaled   by  the   progressive  participial 
constructions   "giving"    and  "suggesting.'^  Another 
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In  one  case,  the  heightened    salience  of  repeated  word; 
derivatives  with  common   roots   ("improvement,"  "sugges- 
tions,"   "suggesting")   may  lead    the  reader   to  conclude 
tnat  he   is  expected   to  contribute  a  description  of  a 
"situation"    (a   state  of  affairs  or  combination  of  cir- 
cumstances)  and   propose   a  process  which  will  provide 
partial   resolution   ("improvement")    to   the  "situation." 
In   the   second   case,   the  reader  may  see  his   task  as 
defining   a   problem ,   a   definitively  negative,   or  prob- 
lematic  single  circumstance,   and   posing  a  definitive 
"solution"    for   "correction"   of  that  problem. 

If   S2  is   considered   ij\   isolation   the  reader  may 
arrive   at   the   following   interpretation.     The  i.nperative 
"describe"   presupposes  the  goal   of  the  prompt   task  to 
b «  a  description.  A  description   is  a   p-oduot  of  personal 
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observation-      Tne   single   instance  of  the  iTiperative 
fom   indicates   that  tne   re.aJer's  observations   are  bsing 
elicited   as   the   central    feature  of  the   task  he   is  asioi 
to   perfor'n.      Presupposed   here   in   the   imperative  for-n  is 
the   salience  of  tnese  observations,   supported   later  in 
S2  by  the  relative  clause  "which  you   feel."      S2  presup- 
poses  that  personally  relevant  observations  are  to  form 
the  basis  for   the  reader's  discussion.     Thus,  the 
reader   is  instructed   to   describe  his  observations.  He 
is  also   to  propose   "solutions**   to   what  he  personally 
considers  problematic,   and  develop  or   illustrate  his 
"reasons"   for   wn y  he   found   his  choice  of  subject  matter 
personally   problematic,   as  per   the   segment  "givinsj  rea- 
sons  for  your  choice/' 

However,    if  the  reader   interprets  S2  in   the  con-- 
text  of  31,    the  prompt   segment  "giving  reasons  for  your 
choice,"   can  be  interpreted  as  "giving  reasons"  why  the 
reader   chose   to  discuss  a   particular   "problem  or  situa- 
tion"  above  all  others  established   in  a  set  of  pre- 
cedents.     In  other  words,   the  reader   is  reminded  that 
he  is  to  be  held   accountable   for   a  background  of  shared 
knowledge,    against  which  he   is  to   justi  fy  his   choice . 
The   reader  may  also   conclude   that  he  is  to  justify  nis 
choice  against  a  background  of  multiple  opinions  as  to 
what  constitutes  a   valid   "suggestion,"  interpreting 
"giving  reasons  for  your   choice"   as   "giving  reasons" 
for  why  the  "problem"   should  be  c:^nsidered   a  "problem." 

Idealized   readings  such   as  the  foregoing  are  always 
exhaustive   in  demonstrating  what  can  be  formally  inter-- 
preted   from   a   text.     Real  readers,   on   the  other  hand, 
may  chose  to   read  selectively   (to  attain  a  unified 
representation  of  all   P^rts  of  the   text  or  merely  for 
the   sake  of  expediency). 

Basically,   what  we  are  suggesting  here,    is  that 
elements  In   the  prompt   pull   the  reader   in  different 
directions.      The   first   sentence   in   the  prompt  might 
lead  a   reader   to  believe   that  he  should  choose  from 
what  has  been   posed  as  a  defined   set  of  suggestions, 
particularly   if  he  has   treated   the  reference  to  sugges- 
tions as  old   information.     Thus,   as  mentioned  above, 
the   phrase   "giving   reasons  for  your   choice"   might  be 
interpreted   as  meaning   "giving   reasons  for   why  the 
problem  was  chosen  above  others."   Other  elements,  on 
the  other   hand,  might  lead   the  reader   to  believe  he  has 
the   freedom   to  cnoose   any  problem   to   discuss  that  is 
relevant  to   himself  (cf., "which  you  feel")    and  give 
reasons  why  the  problem  was   problematic   for   himself,  as 
opposed   to  why  the  problem   should   be   considered  a  prob- 
lem ,   per   se  . 

It  has  become  customary  for  linguists  to  view  tne 
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text  within   the  sitjation  in  wnich  it  was  presenteJ. 
In  linguistic   tarijs.   tne  proupt  represents   a   piece  of  a 
continjous   interactive  "text"   with  the  interpretive 
response  which  is   to   follow.     The  larger   context  of  the 
discourse   is   the  writing  assessment  situation  which 
involves   the  process  of  evaluation.     Because  of  the 
assassTient  situation,   readers  n5ay  interpret  the  pro.-npt 
as  carrying   an   implicit  directive:   conform  to  the 
correct  development  of  a   topic   in   form,   content,  and 
context.     Student-readers  who  interpret  a   prompt  dTf- 
ferently  from  the   teacher-readers  who  evaluate  their 
responses  may  be   unable   to  conform   to   the  evaluators' 
conceptions  of  what  constitutes  correct  interpretation. 
iMoreover.   within   the  assessment  situation,   students  are 
very  likely   to  be   preoccupied   with  the  search   for  the 
right  an  s  wer . 

Analysis  of  Oral   and   ;V  r  i  1 1  e  n  R  esponses   to   the  prompt 

The   second   body  of  data  under   investigation  in  our 
analysis  is   the  students'   oral   responses  to   the  prqmpt. 
In   order   to   compare  our   analysis  of  the   features  of  the 
prompt   with  the  ways   students  responded   to   it  during 
the  "think-aloud"    procedure  conducted  during   the  pre- 
writing  interviews,   we  adopted   a  unit  of  analysis  that 
would   allow  us  to   systematically  compare   tne  students 
responses  with  particular   prompt   segments.  Several 
systems  of  text  analysis  and   descriptions  of  discoj'^sa 
structure  offered   potential.     Much  of  the  pioneering 
work  on   the   composing   process  has   focused   on  narrative 
discourse,    including  descriptions  of  narrative  struo« 
ture   (Handler  and   Johnson,    1977.    Rumelhart,    1976),  and 
systems  of  text  analysis  which  descrioe   features  asso- 
ciated  with  its  production   (Labov.    1972.    Cnafe,  1930). 
Because  we  were  studying  processes  of  interpretation 
and   composition,   systems  of  analysis  which  were  based 
on   (and   derived   from)   actual   language  phenomena  held 
the  most   appeal.     If  the  point   is  to   describe   the  ways 
individuals  actually  do   produce  meaning   in  the  process 
of  interpreting  or   responding   to   text,   then   systems  of 
analysis  that  incorporate  process  and  on   line  produc- 
tion should   yield   the  niost  useful  information. 

Mature  and   Relevance  of  Idea  Units.     For  this 
reason,    the   pre -writing   interviews   ware  transcribed  and 
segmented   into  oral   ••idea  units"   according   to  criteria 
for   oral   language  analysis  developed   by  Dr.  'Wallace 
Chafe  of  tne  Linguistics  Department  at  the  University 
of   California   in   Berkeley  (Chafe.    1979.    1930).      An  idea 
unit  is  basically  a   chunk  of  words  bounded   by  a  measur- 
able pause  and/or   a  change  in   intonation,    which  serve 
as  indications  that   the   speaker   is  treating   the  word 
group  as   a  conceptujl  unit. 
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Several   assunptions  lie  behind   tne  posited 
existence   of  such  a   unit..  For   the  reader  who  is  not 
faniiiar   with  Cnafe's   theory,    they  are  briefly  ojtlined 
here.      Cnafe  suggests   that  thinking   involves  three 
najor   coTiponents:    1)    information,   2)    the  self,   and  3) 
consciousness.     With  regard   to  "information/'  Cnafe 
suggests   that  '♦at  any   given   time,   a   person   has  avail- 
able  a   large   fund   of  knowledge   from   several  sources. 
One   source   is   perception  of  the  world   around  us, 
another   is  memory,   a   third   is   affect-the  emotions, 
feelings  and   attitudes  associated  with  what  we  perceive 
and    remember"   (Chafe,    1930,   p.    11).      Chafe   also  posits 
the   existence  of  a   "self,"   which  he  describes  as  an 
"executive"   who   provides  the  central    control   over  what 
is  happening  "      (Chafe,    1930.   p.    11.).  Presumably,  this 
executive   provides   control   over   where   attention  is 
directed.     Chafe  also   notes   that  this  executive  has  a 
variety  of  needs  and   goals  which  are  associated  with 
central   needs  and   their  attainment.  Consciousness, 
according   to   Chafe,   possesses  a  central   focus  and  a 
peripnery,    and   moves    in  " j er ks "^  an al o g o us   to  saccadic 
eye  movement.      It   is   "highly   limited    in  capacity,"  and 
it   is   "highly   limited   in   the   time  it  can  dwell  oi\  one 
piece  of  information   "   (Chafe,    1930  ,    p.    11.).  Cnafe  also 
suggests   that   there   are  "linguistic   expressions  of 
focuses  of  consciousness,"   and   these   he  terms  "idea 
units."   Caafe   uses  the  idea  unit  as  the  basic   unit  for 
his   system   for   analyzing  text.     Each   idea  unit 
expresses   a   focus  of  consciousness  and   functions  to 
contribute   to   the  development  of  Knowledge  which  the 
speaker   is  constructing.     Each  idea   unit  represents  a 
coTiposite  of  lexical    information  corresponding   to  a 
simple   clause,   or   syntactically,  one   verb   and   its  asso- 
ciated  noun  phrases. 

Procedure   for   Analyzing  Oral   Responses .  The 
tape-record ed   interviews  were   transcriLed   according  to 
procedures  developed   by  Chafe   (1979.    1930).  Tne 
researcher   who   supervised   this   portion  of  the  project 
had   previous   experience  in  data  collection   and  analysis 
using  Cnafe's  materials  and   techniques  in  seminars  and 
research  at   the   University  of  California,  Berkeley. 
Tne  members  of   the   project  staff  involved   in  preparing 
transcripts  of  the   tapes  were  introduced   to    the  basic 
concepts   and   procedures  in  a  training   session  before 
transcription  began. 

The   preparation   of  a   transcript   required   t^at  a 
staff  member   listen   to   the   tape   repeatedly,  concentrat- 
ing  on   one  aspect  of  the  language  for  transcription 
each  time.     An   initial   listening  captured   the  basic 
lexical   content  of  the   interviews.      Tne  oral  discourse 
was   then   segmented   into  "idea  units"    by  using  a  combi- 
nation of  three   types  of  cues:    1)   intonation,  2) 
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hesitation,   and   3)   syntax.   According  to   Chafe.  Intona- 
tion  provides  the  most  consistent  signal  of  the  bou.n- 
darTes  of  such  a  unit  of  discourse.     Hesitations  ara 
also   powerful   indicators  of  a   possiole  boundary  ostw-en 
ideas,   -hile   syntactic   structure   is  tns   least  necessary 
infomation   source   for   such  decisions.      None  of  the 
types  of  cues  alone  guarantees   tne   existence  of  /^^^ 
unit,   but   the  co-occurrence  of  two   is  a   strong  indica- 
tion of  a   conceptual  unit. 

To   prepare   a   transcript  of  each  interview, 
res-arch  assistants  were   instructed   to   follow  a  six- 
step  procedure.     First  .   they  were  directed  to  siiiply 
record   th 2  words,   i.e..    the  lexical   content,    from  the 
tape.     This  step  in   itself  involved   listening  repeat- 
edly to  tne  tape. 

Second,    they   were  directed   to   locate   the  segtents 
of  th-   tape   containing   a  large  h^uber  of  hesitations 
(including  both   filler   sounds,   such  as   urn.   ah.   and  che 
lengthening  of  tne   syllables  of  actual   wor d s ) .     He s i t a- 
tional   sounds  were   to  be   transcribed   phonetically,  witn 
r-p>ated   listenings  recomnended    1  )   to   sequence  these 
correctly  and   2)   to   verify  the  number  of  syllables  or 
beats  of  filler   sound   in   each  hesitational   segment.  in 
the   transcription  system,   a   lengthened   syllable  was 
indicated   by   two  dashes  appended   to   the  syllable. 

Third,   a   related   step  involved   focusing  attention 
on   thl-iT^ty  spaces  of  the  discourse  in  order   to  iden- 
tify the   pauses  and   to   record   the-n   in   relation  to  the 
meaningful  material   in   the  transcript.     Two   lengths  of 
pause   time  were   indicated:    1)    a   slight   break   in  tempo 
and   2)   a   longer  pause. 

Fourth,   the  research  assistants  were  asked  to 
identirrThree  general   categories  of  intonational  con- 
tours:    sentence   final,   clause   final   and   question  into- 
nation      Tne   recommended   procedure  was  to   first  mark 
those  contours  which  stood  out  most  clearly   and  then 
use   tnose  contours  as   reference   points   against  wnich  to 
judge  other   intonation   shifts.     The  distinctive  falling 
Ditch  of  sentence   final   intonation,   for   example,  is 
generally  more   easily  identified   than  clause-final 
intonation. 

Fifth, atyped  draft  of  the   transcript  was  prepared 
in   wnich  sach  idea  unit  appeared  on  a   separate  line. 
The   idea   units,   spoken   by  the   interviewee,   were  num- 
b-r-d  consecutively  and   any  section  where  co.nments  by 
tin-' interviewer   and   interviewee   overlapped   was  indi- 
cated  by  brackets   at  the   points   where   the  overlap  began 
and  ended. 
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■narks,   such   as  coTinias,   dashes  and   periods,  but 
discourse  con taining   taese   signals  can   also  be  given 
different  readings  b y     i f f e r en t  r e a d er s  on  separate 
occasions.      These  signals  are  ultinately  able   to  serve 
as  approxiTiate  guidelines   for   a   reader's  conprenension 
of  the  tej(t,   but  theV  cannot  record   the   rhythti  and  pat- 
terns  whicn  characterise   the  act  of  production. 

Furthermore,   students  who  are  learning  to  coiiposa 
in   the   written  mode  often  have  not  mastered   the  conven- 
tions of  writin-g.      Tney   frequently  do   not   provide  the 
punctuation   signals  wiiich  cnaracterize  mature  prose, 
and   therefore,  -punctuation  cannot  be  used   to  guide  th,e 
analysis  of  studentwritingin   the   same  ways   that  it 
might  be  used   with  SKilled   adult  prose.      Instead,  we 
used   a  combination  of  gr^-nmatical   and   semantic  cri- 
teria,  including  recognition  of  shifts  in   point  of 
view.     The   criteria  ar-e  given   in  Figure  5. 

Fi  gur  e  4. 


Criteria  Fo  r .  Segmenting  W£i t t£n  Discourse 

In   segmenting  written  discourse   into   idea  units: 


A  .        Do   se  pa  r  a  t  e; 

1.  a   segment  ending  with  a  period. 

Tnis  means  that  even  clauses  and   phrases  are 
tr^eated   as   idea  units  when  they  end   with  a 
period.  '  . 

2.  a   segment  ending  with  a  dash. 

3.  a   run-on   sentence   into  separate  independent 
clauses. 

a  compound   sentence  into   separate  independent 
clauses,   even   if  the  subject  is  deleted  in 
one  clause. 

5.  a   pre- posed  or   subordinate  introductory 
cl ause  » 

e.g..   Because  he   had   to   leave  early,   //  John 
massed   the  main  speech. 

6.  "    anon-restrlctiv eel ause. 

7.  directly  quoted   dialogue   from   the  speaker 
tag^ 

e.«.,   "XXXX,"//said  John.^ 
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Sixth,   a   second   researcher  also  list.enei  fco  the 
intervie,^,   validating   the   transcription  and  decisions 
about   segmen ta tian  of  the   discourse.     Any  discrepancies 
were  resolved   by  a   third  judise. 


Tne   symbols  used  in 
su-n.Tiarized   in  Figure  1. 


the   transcription  syste.-n  are 


Figure 


Clause-final  Intonation 
ContouTf  rising  or  falling 
p  i  tcht    l-npr  e  55  i  on  of 
I  ncoTip  1  ctencs5 

Sentence-final  intonation 
nistinctively  falling 
pitch.    Impression,  of 
compl e  tenes  5 

Question  Intonation 


a  Dreak   in  rhythut  too  " 
brief   to  be  jpcasured 
(Darely  notice«tble  pauset 
a  slight  break  .In  tempo) 


k  oause 
itea  5  jr  e 


long  enough*  to 


Lcr>c3t  hen  ing 
syl 1  able 


of  sound  or 


P 

Tne  wr 
fo  r  an 
Cnafe  ' 
wr i  1 1  e 
tio  n )  . 
pre  sen 
for  se 
tne  t  r 
of  con 


rocedur 
i  1 1 ?n  c 
a  1  y  s  i  s  • 
s  adapt 
n  disco 
Ho  we  V 
t  a  n  a  1  y 
gTi  en  t  a  t 
ea  tm  en  t 
Joined 


e  for  Segmen 
OTipo  si  tions 

Tne  system 
ation  of  his 
urse  (Cnafe 
er ,  the  cri  t 
sis  differed 
ion.  These 

of  s  ubo  rd  in 
verbs  and  de 


^Iimm  Compositions . 
were  also  segmented   into  units 
of  segmentation  was  based  on 
oral  discourse  techniques  to 
and   Daniel ewicz,    in  prepara- 
eria   finally  used   in  the 

slightly   from  Chafe's  rules 
differences  primarily  involve 
ate  clauses  and   the  phenomenon 
leted   segments  in  sentences. 


Two  major   problems  were  encountered   in  finalizing 
a  set  of  criteria   for   the  segmentation  of  the  written 
compositions.     First,   existing  criteria  for  segmenting 
oral   discourse  are  heavily  dependent  on  indications  of 
process:   pause  time,   hesitations,   false  s t a r t s  ,    i n to n a- 
tlon,   etc.    (Cnafe,    1930  )..^   Such   signals  of  process  are 
non-existent  in   a   piece  df  written   prose,   unless  a 
recori  of  its  production,   such  as   that  provided  by  a 
videotape,    is  available. 

Second,    the   signals  used   in   written  prose   to  indi- 
cate where   pauses  or   intonational   contours  might  occur 
are*" often  approximate  and   therefore,   ambiguous.  Writ- 
ers  Indicate   pauses. of  various  lengths  wltn  punctuation 
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8. 


a   rhetorical   question   fro'n   its  ans^^er. 


9.  direct  representations  of  sounds, 
e.g.,   "Beep!  " 

10.  sefj-nents  containing  conjoined  verbs  if  there 
is  modification  after   the   first  verb. 

11.  a   restrictive  clause   wiich   introduces  new 
in  forniation  . 

12.  a  clause  which  conveys  a  shift  of  spatial  or 
temporal  orientation,  or  which  sets  up  a  new 
scene. 


13. 

a 

clause   which  signals  a 

proces 

S  0 

f  recall. 

e  . 

g.,   "if   I  re. Tie. Tiber  cor 

rectly , 

// 

•    •  • 

14. 

3 

clause  which  signals  p 

er  sonal 

In 

teraction 

in 

the  in  terview,  i.e., 

which  m 

a  ke 

s  a  text 

e  X 

ternal  reference. 

15. 

a  n 

evaluative  comment  on 

som  e  t h 

ing 

in  the 

t  e 

X  t . 

Do 

no  t 

separate  : 

16. 

a 

restrictive  clause  whi 

c  h   i  3  n 

0  t 

used  to 

in 

trodune  new  informatio 

n  . 

17. 

a 

non-preposed  subordina 

te  clau 

se  , 

unless  i 

in 

dicates   a  shift  of  teii 

po  r  a  1  0 

r  s 

p  a  t  i  a  1 

or  i  en ta  t ion  . 

18.  an   introductory  phrase,   or   any  other  phrase 
structure . 

19.  the  subject  and  verb   in   a  clause. 

This  :neans  that  idea   units  may  be  embedded 
either  between   a  subject  and   a  verb,  or 
between   a  subject-verb  construction  and  its 
comj)lement  without   separating   that  subject 
and   verb   into   separate  idea   units.     The  suo- 
ject  and   verb   are  s im pi y  1  a b e 1 ed   with  sub- 
scripts of   (a)    and    (b)   in   add^ition   to  the 
number  of  the   idea  unit, 

e . g * ,  *"He,/I   thlnK,    stole  one  of  the  two 
baskets  of  pears."   i    1a/2/lb  "He  stole,/! 
think, /one  of  the   two  baskets  of  pears."  = 
1a/2/lb 

20.  Clausal   structures  wnich  are  complements  to 
hi  g  he  r  V  er  b  s  , 

e.g.,   "I  thiniC  he  stole  one  of  the  two 
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basicetsofpears."   s    1  lU 

"Ha   bonier ei  what  happened   to  the 
paars."  '  =    1  lU 

2'1  .     an   apositlve   fro.^   the  noun   to   i^nicn  it  stands 
in  apposition. 

22.  ind  irec t  speech, 

e.g.,   "he   told  me   Jane   was  the   wir>her."   is  1 
lU 

23.  infinitive  phrases  in  series  when  they  appear 
in   the  conplenient  to   the  verb. 

24.  c2or. joined  verbs  if  th?r^  i?  no  i!!Ojiric?tion 
after  the  first  verb,  especially  if  the 
activities  are  similar, 

e.g.^"He   pushed  and   shoved   the  door  open."  s 
1  lU 

We  purposely  have  not  called  our  unit  of  analysis  for 
tne  written   compositions  an  "idea   unit",   because  we  did 
not  collect  on-line  production  data  in   the  written 
mode. 


A  Typology  of  Idea   un  i^t  Functions 


In   order   to   exaTiine   the   p-aths  o*f  interpretation 
taken   by  each  student,   a   typology  of  functions  was 
developed   so   that   the   functions  of  each  idea  unit  in 
the   interviews  could  be  characterized.     The  typology 
was  "modeled"   on  the  typology  of  functions  which  Cnafe 
developed   for  narrative   (Chafe,    1930).     To  identify 
patterns  of  strategies  that  extend  over   several  idea 
units,    the  analysis  superimposed   ideas  frO:Ti  Labov's 
Studies  of  Black  English  Vernacular  Narratives  ,    (  1972). 

Chafe's  typology  of  idea  unit  functions  Is  based 
on  the  way  they  function  as  pieces  of  wnol e  commun ic a- 
tions.      In  discussing  his  pro c ed ur e s " f or  determining 
the   functions  of  idea  units.   Chafe  notes   that  the  fol- 
lowing question  was  asked:     "In  what  ways  does  the 
.focus   of  consciousness  which  this  idea  expresses  con- 
tribute to  the  development  of  knowledge  wnich,   focus  by 
focus,    the   speaker   is  constructing?"   The   present  study 
required   students   to  perform  a  more  complicated  task, 
than   the  "knowledge  con  st  r  uc  t  itig"   task  Chafe  posits. 
Our  own  task   involved   a  two  part  process:    1)   a  response 
to  a   stimulus,   and   2)   a  plan   for   composing.  Therefore, 
the  question  whicn  guided  our   analysis  was:   "In  what 
ways  does  this  Idea  unit  contribute  to   the  development 
of  an   Interpretation  of  the  prompt,^   or   to   the  composing 
plan  the  speaKer   Is  constructing?" 
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In   applying   this  procedure  of  quastionin?  the 
function  of  each  idea  unit  in   the  oral  interviews, 
founJ   that  students*   confinents  demonstrated   their  aware- 
ness of  their  "roles"    in   three  different  "worlds."  In 
talking  with  the  researcher,   the  student  was   the  coTnion 
denominator   linking   the  "present"   interview  situation, 
the  student's  "past"   experience,   and   "future"  perfor- 
mance  in   the  assessnent  situation.     Thus,   the   task  in 
our  study  differed   from  that  of  Chafe,  who  character- 
ized  his  own   subjects  as  being  "peripherally 
consc ious .  .  .of  at  least  two  quite  disparate  worlds: 
that  in  which  they  found   themselves   sitting  across  from 
an   interviewer.. .and   the  wo  rid   inside   the  film...,"* 

Within   the  "world"   of  the  interview  idea  units 
served   two  primary  functions:   interaction  and  reference 
to   the   "think-aloud"   process   (i.e.   the  process  of  ver- 
balizing a  response  to   the  prompt).     It  could   be  argued 
that   in  one   sense  all  of  the   idea  units  within  the  oral 
interviews   served   the   function  of  interaction,  because 
we  were  dealing  with  an   interactional  situation, 
i^ithin   the  world  of  the   interview,   however,  certain 
idea   units  were  identifiable  as  "overtly"  interac- 
tional.     In   general   these  idea  units   fit  into  a  frame- 
work where  an  expectation  for   a  response   from  the 
interviewer   existed,   such  as   a   question/answer  format, 
or  a   framework  wnere  a   student  acquiesces  to   a  direc- 
tive.     Tnese   included  many  different  types  of  interac- 
tion,  and   ranged   from  single  word  comments  to  full 
clauses   in  response  to  questions. 

Interactional. 


I*  ...  tell   nc  what  you  thfnn  yDu^d  have  to  do 

In  resoDnding  to  that  particular  topic? 
S:  Ok.  (Pa<n) 

IS  Well  ifhat  do  you  think  you^ll  have  to  do? 

If  I  would  •  .  write  about  It?  (Chip) 

Think-Aloud  Process. 

S:  .  .  .  urn  .  .  •  <sigh)  hu-n — .  .   .  That's 

about  all  I   can  thin<  of  I  thiak»  (Carla) 


A   second   wo rid   in   which  the   students  were  engaged 


•For  tne  reader  wn o  is  unfamiliar  with  Cnafe's 
analysis,  we  note  that  Cnafe  collected  interview  data 
based  on  subjects'  recall  of  subjects,  actions  and' 
events  within  a   filmed  narrative. 


ERIC 


44  ,^25 


^33  tti-  world  of  assessment.     Co^nen  ts  rel  atad  to 
assessT^ent  had   a   future  orientation  signaled   by  th3  use 
of  tne  condition?.!,     Many'of  these  comments  involved 
planning   for   the   co'nposition  which  the   student  was 
about   to   write;    therefor-:-,   several  of   the   idea  units 
w^re  categorized    as   serving  the   function  of  signaling 


identified   within   this  gen- 


organ  i  za  M  o  n  a  1   strategies  or   tne   sequence  of  content  ^ 
being  planned   for   the   essays.     A  wide  variety  of  func- 
tions  could   potentially  bs 
e?"al  grouping. 

Planning   for  Composing . 


US)   ss  ...  Hovi?  would  1  30  about  writing  rt?  Maughs) 

(16)  ...  Proba:>ly   #  .  .  act  lIKet 

(17)  ...  urn  .   .  i^rlte  It  more  like  a  Jtory.  (Kin) 

(^9a)   S:  ...  And  then  my  third  paragraph  .  .  urn,  (Kim) 


ilZO)  S: 
(IDla) 

(1D2) 

aoib) 


.   .  I'd  re  •  .  read  through  It  again* 

see  }f  I  could  make  It  tighter  .  .  In  any  way  . 

by  deletlngr 

.   .  you  kno'^r 

...  i^orrfs  here  or  there#  ^<lm) 


Still  anotne 
tne  world  of  tnei 
and  background  kn 
The  interview  ser 
called  upon  previ 
preparing  for  the 
exhibited  various 
reviewing  ^'past'* 
which  might  be  re 
si  t io  n  .  Again,  a 
the  f  u  t  j'^e  ,  shift 
by  shifts  in  verb 
form  3  . 


r  "world''  whic 
r  own  personal 
owledge  in  res 
ved  as  an  aren 
ous  knowledge 


upcoming  as  se 
styles  of  rec 
.•^perienceas 
levant  to  the 
s  In  planning 
5  in  "worlds*' 
tense,   in  thi 


h  students  accessed  was 

experience/observations 
ponse   to  the   prompt . 
a   in  which  the  student 
and  experience  in 
ssmen t .     The  students 
all  in   the  process  of 
"potential"  material 
prompt   and/or   the  compo- 
for  the  composition  in 
were   primarily  signaled 
s  case,   to   past  tense 


Past  Experience  . 


(27)       .    ,   .   and  1   had  a  ca  .   .   I  had  my  srhecule.  (Don) 

(2C)  •  •  wrll  1  c'n  remerrbcr  when  they  built  this  schocl* 
\  21)       •    •   •   and  .    •  *»e  carre  over  i    (Bob ) 


In   discussing   the   three   "worlds"    in   wnich  the  stu- 
dent operates,    1)   present  interview,    2)   future  assess- 
ment,  and   3)   past  experience,   we  have   introduced  four 
of  trie  major   functions  which  any  single  idea  unit 
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appeared   to   serve:  Intera 
planning   for   co.-nposin?  an 
exp3rience.      In  addition 
aoove,   W2    identified  two 
w^ich  individjal    units  20 
knowledge,    a 'id  reference 
section,    tnese   two  functi 
cussed,   and   we   will  Intro 
evaluation.     We   should  no 
Chafe,   we   found   that  a  5i 
more  tnan   one  function  si 
iJea   units  which  served  a 
which  referred   to   the  pro 


ction,    reference   to  process, 
d   reference   to   past  (personal) 
to  the   functions  described 
other  main   types  of  functions 
uld   serve:   Reference   to  sharej 
to   tne  p'-ompt.      In   the  next 
ons  will  be  described   and  dis- 
duce   the   global    function  of 
te,   at   this   point,   that  lif<e 
ngle  idea  unit  could  serve 
nul tan eousl y •     Many  of  the 

double   function   were  those 
Tipt  . 


Reference   to   the  Prompt,      In   examining   the  data 
we   discovered   chat  many  of   the   idea   units  incorporated 
words  or   groups  of  wprds   fron   the   prompt.      Reference  to 
tne  prompt   was  generally  accomplished   in  one  of  three 
ways:    1)   by  the  mention  of  the  exact  wording  of  seg- 
ments of  the  prompt,    2)   by  a   parapnrase  of  segments  of 
the  prompt,   resulting   in    the   reconstruction  of  the 
wording  of  the  syntax     of     the   prompt,   and   3)  refer- 


enc 

e  to 

tne 

who 

1  e 

prompt .      In  a 

11  cases 

.   r  e  f  e  r 

en 

c  e  to 

the 

pro 

m  p  t 

ofte 

n 

served   as  a  po 

int  of  d 

e  pa  r  tur 

e 

wh  i  c  h 

res 

ul  te 

d  in 

ide 

a 

units  serving 

multiple 

functi 

on 

s  to 

su  i 

t  th 

e  pu 

r  po  s 

of  the  student 

at  any 

given  i 

n  s 

t  a  n  c  e 

in 

the 

i  n  t  e 

r  V  ie 

w . 

In  referring 

to  the 

prompt. 

s 

tudents 

bo  r 

r  0  we 

d  an 

d  pa 

r  3 

pn r  a  s ed  units 

of  di  f f e 

rent  si 

ze 

s  . 

Som 

et  im 

e  s  s 

ingl 

e 

words   fron  the 

prompt 

appeare 

d 

in   s t u - 

den 

t  CO 

mm  en 

ts. 

often   entire  phr 

a s  e s  or 

sen  tenc 

es 

were 

adopted • 


^^^^1  ^ord  ing : 

The  student  reiterates  the  exact  wording  of 
the  prompt,  or  a  segment  of  the  prompt,  and 
may   incorporate   it  into  his  own  statements. 

(2)  Ck   (laughlngt   long  pause)   •   •   .'kaY»  dcscrTbe 

one  problerrt 

(cf.  ■Describe  one  problem*) 

(4)  •    .   .  Describe  one  problem  or  situation  at  Central 

which  you  feel  needs  correction  or  Improvement. 

(KJrr) 

(cf.  "Describe  one  rroblem  or  situetlon  at  Central 
which  you  feel   needs  correction  or  Irrpr  overren  t  .* ) 

t?)  And  I  do  n't  knew  an  •   •   of  any  .   ,  ,  suggestions 

of   •    •   •    for    Irrprovement  • 
(cf,  "sugcestlons  for  ifrprovenent  * ) 
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Paraphrase  : 


Paraphrase  Involved   tna  r eson s t r u = t i o n  of  th 
wording  or  the  syntax  of  the  proTipt  tnrouin 
s Jbsci tucion  ,   deletion  and   reordering  of  wor 
groups  of  varying  size. 


(^n  Just  describing  try  prcbUfr.  (Kirr) 

(cf.  "Descrlb-  one  problem") 

Urr  .  .  .  ck,  student  apathy  is  a  problem  at  Central, 
Z-.ir)     (cf.  -one  problem  ...  at  Central-) 

1^1  -Describe  cne  problem  or  situation  at  Central 

*  whUh  you  feel  needs  correction  cr  improvement. 


(5) 
(6) 


(33) 


.  1  czr  think  of  one  (lauohs)  , 
.  The  lunch  lines  arc  horrible.  (Katiei 


,iri  ...  and  -sugcest  cne  or  irore  solutions. • 

^ell   .   .".  I  v.ou1d  think  they  just 
:   :  ihey   should  make  a  bigger  cafeteria.  (Kat.e) 

An^  ahcut  a  solution  to  the  prcblem 

\  1  lllsl  \.l   .   .  the  soluticr  tc  .y  problem 
would  be    .  .  .  peer  pressure,  (f-ert.n) 

.   .   .  there  really   Isn't  any  solution  to  .   .  the 

^cf''-D;s!Mbe  one  problem  .  ■.  .  cWing  ...  one 
or  more  solutions.") 


Reference   to   t h e   prompt   as  a   whol e : 

...  I  feel  this  .   .  ur  .  .   is  asking 


(2)  .   .  .  um 

me  to 

(3) 
(9) 


r.  .  you  know  .   .  ten    .   .  the  problems  of 
Central.  (Carla) 

It  just   ...   It  just  .    .  bring  Central   to  re.  (Joe) 


Frequently,   references   to  the   prompt   w^re  followed 
by  references   to  personal   ex pe r i e nee ,   r e f er en c es  to 
shared   knowledge,  or   statements  of  opinion.     The  typi- 
cal   pattern  involved   a  mention  of  a   phrase   from  the 
prompt   followed   by  related  commentary. 

Typical   Pattern:     Reference   to  Pj;^mpt / Comme n t 


Ide  8 

Katie 

Reference  ♦   -Describe  one  probl  ert  or  s  ituat  ion 

prompt:  -t  Central  which  you  feel  needs 
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correction  or   i  rproverr  er  t 
Cofpment:  5    •  .   .   I  car  think  of  cne  (laughs) 

6  •  •   •  The  lunch  lines  are  horrible. 

Reference  10   •  •  and  "suggest  one  or  rrcre  solutiors." 

CorrrFcnt:  11   .  .  .  ^lell    ...    I  would  think  they 

just   •  •  they  should  n^eke  a  bigger 

caf e  te  r  1  a  • 


Mart  In 

Pefe  rence 
to  prorpt' 

Comment ' 


17  Ind  .   .  about  a  scluticr  to  the 
prob1  err 

IB   ...  I  guess  the  •    •  the  solution  tc  my 
problerr  would  be   .   .   .  pe«r  pressure. 

19  and  you'd  have  to  .  •  •  5*^  •  •  9*^ 
kids  Involved  In  wantlnc  the  school 
tc  1 ook  n I ce • 


Patterns  of  responses  associated   with  reference  to 
the  pro-npt   in   the   interviews   allowed   us   to   exa-Tiine  the 
variety  of  interpretations  given   to   seg-nents  of  the 
proTipt.     Tne   exanples  below  illustrate   the  variety  of 
ways   students  responded   to   th3   first   sentence   (S1)  of 
the   prompt   in   accessing   (or   atte.Tipting   to  access) 
shareJ  knowledge. 

Reference   to  Shared   Knowl ed  ge 


(2)  Urn  .  •  that  cojjes  to  mind  hoti  everybody's  al««ays 
talking  about:     "All  we  need  Is  grass!" 

(3)  You  kno.*»  "There  should        a  $#rflmming  pool!" 
(^)  'Cause        v^sre  supposed  to  have  a  like  uh  a 

football  stadium. 
C^)  He  were  supposed  to  have  •  •  •  a  SMlnmlng  pool t 

(6)  bo»rfl Ing  a1 leys  i^^) 


»  a  a 


(70)  •   *  •  Dkay  .   .   It  says  'Many  different  suggestions 
for  I uprovef^ents  .  .  of  Central    .   .  have  recently 
been  iiade •  ' 

(71)  •   •  .  so  >   •  like  •   •  you  aean  just  from  students. 

(Carl  a) 


They  are  thinking  about  .  •  urn 
(5)  •   •  •  giving  aaney  far  #  .  tD  nake  the  school  looic 

a  lltMe  better. 
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(6) 
<7) 
(6) 
(9) 
(10) 


And  then  — 

•  •  .  they've  .   •  they've  got  a  big  granti 

•  •  .  state  or  ftdsral  or  $o^ethirg» 

•  •  r  to  fmprove  the  currlculuTi 

•  •  •  of  the  school.  (Martini 


II  »  » 


(8} 


♦  •  •  and  I  dori't  know  an  •  •  of  any  sugqcsticns 
of  a  •  for  improvc^»ent* 

•  •  •  that've  been  made  recently.  (Don) 


i  a  ft 


(?) 
(3) 
(«•} 


«   •  »  uh--  •   •   •  the  I-nprovcTients  of  Central. 

•  •  •  iP^cll   one  thing  has  been  the  landscaping* 

•  •  •  It  •  .  like  it  used  to  be  mud  all  over  and 
they  started  to  put  trees  —  • 


C7) 
(&) 


•  •  •  un  •   •  •  they've  painted  the  doors  do^n  there. 

•  •  •  and  they've  kind  of  just  changed  the  i^ay 
they've  uh-- 

•  •  «  taught  the  •   •  uh  •  •  students  fron  last  year* 

•  •  •  last  year  was  a  dlffzrtnt   •  •  -neth^d  they 
tau^^t .  (Jay) 


(ID) 


So-ne  of  tne   stujent^s   in   this   group   iniicatei  they 
were  a»^3re  of  suggestions  tnat  had   be^n   made  (e.g., 
Alice).     A-nong   fiese  students,   however,   the  suggestions 
were  attributed   to   various   soaroes,    (cf»    Carla,  Mar- 
tin).     A.-nong   the   students  who   indicated   they  had  sotiq 
knowledge  of  suggestions,    two   indicated   they  had 
knowledge  of  specific  suggestions  which  fit  the  ten- 
poral   constraints  specified   in   SI.      These  suggestions 
had   been  detailed  in  a   school  newspaper   article  jis-=' 
cussing   the   tentative   plans  which  school   personnel  were 
drawing  up   to   use  recently  acquired   grant   funds  for 
school    im  pr  ov  eji  en  t  projects   (e.g.,  Martin). 

In   contrast   to   students   who   claimed   to   have  sons 
knowledge  of  what   the   first   sentence  could   refer  to, 
other   students   Indicated   a  lack  of  such  knowledge 
(e.g.,    Don).     Another   student,   who   had   ignored  the 
first   sentence  altogether  in   his  first   interview,  made 
the   following  comment  wnen  he  was  questioned   about  wnat 
he  didn't  like   about   the  topic: 

Just   the  way  it's   put  together    ,    .  .  . 

un    .    .    .'cause    I  haven't  heard   very  many  of 
tne  s    .    .   Central   High  School's  suggestions  o 
r    .    for   im pr o V  em  en t / I  wouldn't  know  mucn 
about  What  to  write  about. 


The   students  clearly  had  different  degrees  of 
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f3T]iliarity  ^itn   the  recent  debate  about  specific 
sujgestioas   for   school       pr  o  v  e.n  en  t .     Their  comTients 
illustrate   that  various   interpretations  were  nade  of 
tne   first   segTient  of  the  pronpt.      In   providing  these 
particular   exa-nples,   we  are  not   intending   to  suggest 
that  proTipt   authors  should   be  held   responsible  for 
insuring   that  everyone  being   assessed   share  exactly  che 
saTie   Knowledge  or  background   information   to   answer  a 
given  proTipt.     Clearly,   that  would   be   an  inpossLble 
tasK.    It   was   appropriate   for   the   authors  of  this  pro-npt 
to   assuTie   that  everyone  would   have  something   to  say 
since  it   is  a   pronpt   wnlch  generally  draws   on  knowledge 
of  conditions  at   the   school.     However,    the  student 
responses   do   indicate   that   the   first   segment  of  the 
prompt  was   read    in  different  ways,   depending   upon  the 
nature  of  specific   knowledge  shared   between   the  prompt 
authors  and   the   eleven   students.      Moreover,    the  student 
responses   revealed   that  this  particular   issue  did  cause 
an  unanticipated   range  of  i n t er pr eta ti on s •  Our 
analysis   suggests  that  the   interpretive   problem  lies 
not  In   tne   fact   that  some  students  didn't  know  about 
the   suggestions   that  had   been  made,   but   rather   in  the 
wording   and   the   lack  of  specificity  in   the  prompt 
itself. 


In   addition   to   tne  various   interpretations  given 
to   sentence  one,    patterns  of  responses  revealed   that  a 
variety  of  interpretations  were   given   to  other  segments 
of  tne  p'^o-^pt. 

5one   students  in   tnis   t^get   group   justified  their 
Qh3i2e  of  a   problem   in   relation   to  othe»"  problems. 


.  that  1  rtould  slnglt  that  ojtt  (attitude) 

•  and  «  •  and  ah  •  •  •  the  ah 

•  the  grounds* 

the  •  •  TandscapSng  and  stuff* 

•  that's  not  jo  Important.   HaCiJU.  < latrCiLLCLH) 


IP. 


r..^s  written  essay,    .Martin  incorporated   the  same  idea: 


ilt)     And  what   Is  landscaping 

U7)     just  a  superficial  cover  that  ^duH  be  destroy  by 
student  with  bad  addftude.     tlOLlJlL  (LllAX) 


Other  students  justified  their  choice  in  terms  of  personal 
relevance: 

{2^)     •  •   •  And — the  reason  I  chase  It  )«as  because   In  the 

beginning  It  i^as  realTy  bothering  me  when  I  was  studying. 
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In  Jay's  wrlctan  essay,  tne  sane  idea  is  sxpandeJ  to 
a   oonssqjenc-?,   and   a   reason   for   tnat  sonseqaance. 


i  n  c  1  J  i  e 


(9a)     The  reason  I  choose  this  problem  Is  because 
ilO)     '^en  1   first   stated  high  school 
(9t>)    my  grades  dropped  really  fast. 


(1^)     1  could  not  concentrate  because  thef^-  was  always  a 

coTimotlon  or  noise  that  distracted  my  attention  frsTi 
the  classroom.  Ja^  (LSiiX) 


Other   students  appHared   to   interpret   tne  pMrasa   as   a  directive 
to  "give  reasons  wny  a  solution  is  needed"   (i.e.  wny  tr.e  probleii 
should   be  considered   a  probleTi): 

<S)     .  .   •  Um  .  •  .  ths  reasons  for  »y  choice  would  be  •   •  . 
bee  ause 

C9)     •  ©   •  the  school  f  ^  flntertflerf) 

C2D)     .  o    It  d«>ssn't  To^k  9006  this  way.     ^  \aiL££^« 


In   Bob's  i^ritten  essayo   ^^"^is  san?  ijes   is  stated  more  for-nally 

ibl     The  •anlandscaped^  look  the  west  side  of  the  school 
currently  ^as  detracts  froTi  the  school    m  ^any  «ays. 


illLx  ilLii^  Factors .      Reading  ability 

snojld   also   be  considered.     One   student   for  exaiiple, 
focused  on   a   fragment  of  the   prompt,   and  treated 
"i-nprovenen  ts"   as   tne  subject  of  the  sentence   in  recon- 
structing  the  actual   structure  of  the   first  sentence. 
iinen  referring   to   SI,   the  student  said: 


.    o    un-^-   •    c    .   the  impr  ov  e-nen  ts  of  Cen- 
tral./   o    o    o   well  one  thing  has  bean  the 
l^ndsc  aping./ o    «    .    .      it    .    .    lii<e   it  used  to 
be  Tiud   all  over   and   they  started   to  put 
trees--./.    0    .   u'n   .    .    -      they've   painteJ  the 
doors   down  there./.    .    .     and   they've  kind  of 
just  cnanged    the  way  they've  uh  —  /.    .  . 
taught   the    .    •   uh    .    .   students   fron  last 
year./    .    .    •    la^t  year   was   a   different    •  . 
met  nod    they  taugnt.*' 


Stujents  such   as   this  one,   wno  have  basic  reading 
problems,   are  also   lively  to   1  a - k  tne  ability  to  "real 
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53t^3  2ri   tne   lines"   and   to   second   gu3ss   th*  t^st 
autlior's   unstatsd   as  s     p  t  i  o  n  s  in   relation   to   the  p'-o-npt 
(Hoetker,    1932)-      Int2rp!-et3tion   depands  heavily  on 
inferential   ability.      If  a   test   is   to   present   a  fair 
and  equal   cnallen^je  to   tne  writing  skills  of  all  stu- 
dents,  a-nDi^uity   in   wording  nfiust  be  avoided,   and  issues 
of  shared   knowledge  should   be  considered. 

Evaluative  and   Marr ati  ve   Functions ,      As  noted 
above,    aTTr   typology  was  Tiodeied   on   tne   typology  of 
functions   wnich   Cnafe  developed    for   narrative  (Cnafe 
1930)   witn   Labov's  worK  on   evaluation  superitiposed 
(Labov    1  972  ).      /^nile  both   Labov    and   Chafe  incorporated 
evaluation   in   their   analyses  of  narrative  structure; 
Labov  placed   more  e-nphasls  on   this  global  function, 
Cnafs   focused   on   tne   firsc   tnree  of  his   four  main  func- 
tional categories: 

1o       personal   interaction   between  th^   spaa^cer   and  his 
0^  ne^  audience, 

? «       p  ^  -3     s  s  ^  s  of  r  e  ?  a  L I   3       J2h  , 

^] ,        tne  r  e  ^  3 1 1   of  a   n  a  r a  t  ^  v  e   a  s   -i   series  of  s  t  :i  t  a  s 
3 n  3    events,   an  ^ 

Cn^ife's   data   in^/^-lvei   the   *-e?all   of  a  narrs- 
tL^'^,         expanded   tne   t  n  i  r  i  a  a  t  e  -5 o  v    3 d  d  v  e  to  include: 

o  t^i'^   e  St,  3D  1  X  sn-n  en  t  of  a  seating, 

,^0  t    n  p  'J  r  a  I   o  ^  2.  e  n  t  a  n  i  o  n  c 

f^  o  I t  r  0  J  J   i  n  g  a  p  3  r  s  o  n  » 

o  "2  '1      i^f^r  izing  a  person, 

5  „  I  J   a  1 1  n  r;  a   pars n         an  0    j  e c  1 0 

b.  en^,  li^l.^i   a  P'3rs3o   in  a  Curative  background 

enga^xr^?   a   psrson   i^i   a  pjnctu^i    e^ent„  and 
'3  e       r  e    n  t  r  0  1  u    i  n  ^  a   p  3  r  s  J  n  . 

Given   tnat   Cnafe's  data  dealt  witn   tne  recall   of  a 
narrative,   his   extended   categorization  of  the  function 
associated   witn   the   introduction  of  characters  and 
tn^ir   enga^eTients   in   a   series  of  states  and   events  is 
clearly   appropriate   to   his  data.     Narrative   forn  and 
purpose  revolve   around  characters  and    their  activiti-s. 
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Labov,   in   contrast,   e.Tiphasizes  the  role  of  ev.alua- 
tion   in   his  own   analysis  of  narratives  of  personal 
experience.     Labov   points  out   that  evaluation   is  the 
means  used  by  the  narrator  "to   indicate   the   point  of 
the  narrative,    its  raison   d'etre:   wny   it  was   told  and 
wn^t  the  narrator   is  getting   at   "    (Labov    1972).  Thus 
evaluation  can   also  be  seen   to   have   a   significant  func- 
tion  wit n in  a  narrative. 

The   student   interviews   in   our   own   data   are  sLiiilar 
in   soTie   ways   to   those   collected   by   Chafe  and    Labov  in 
tnat   they   involve  the   recall   of  personal  experience. 
Because   they  involve  a  recounting  of  experience,  they 
contain   both  narrative  and   evaluative  elements. 

However,   the  Interviews   we   collected   from  the  stu- 
dents also  differed   in   Important  ways   from   those  of 
Cnafe   and   Labov.      Perhaps   the  major   difference  is  a 
difference  in   overall   function.      Briefly,   the  over-all 
task  of   evaluation   in   the  narrative   situation   is  to 
establish   that  a  given  story  is   worth   telling,   or  that 
a  given    fact  or   event   is   indeed   remarkable.      The  "bur- 
den  of  proof"    that  his  story   is   a  good   one  lies  with 
the   person   telling   the  story.      The   "burden  of  proof"  in 
the   case  of  our   students,   however,    involves   more  than 
evaluating  an   experience   to   show  that   it  is  "interest- 
ing."  Tne  nature  of  the   student's   task   in  responding  to 
the   prompt   in   this   assessment  ultimately  involves  the 
projjction  of  another  genre:    persuasive  argument.  In 
particular,    it   involves   the   framing  of  an  argument  in 
sucn   a   way  as   to  answer   one  or  more  of   the  following 
questions:    (cf.   pronpt  analysis   above)    1)   why  the  stu- 
dent chose   a   particular   problem   above  other  possible 
choices,    2)   why   the   problem  was^geney ally  relevant 
(i.e.   why   the   problem  was  a   problem),    and/or   3)  w^^y 
problem   was   personally   relevant  .      In   the  students^'^  pro- 
ductions,  both  oral   and   written,   these  "arguments" 
appear   in   the   form  of  discussions  of  why  certain  prob- 
lems  are  more  worthy  of  consideration   than   others.  The 
nature   of  the   students'    task   required   an  evaluative 
"frame"    for   the   purpose   of  argument   to  which  reference 
to   personal   experience  could   be  applied.  Thus, 
although  "narrative"   does  occur    in   the   form   of  the 
recounting  of  personal   experience   in  our  data  (incor- 
porating  the   sort  of  development  analyzed   by   Chafe  and 
Labov),  narrative   elements  generally  served  secondary 
illustrative  functions. 

In   exaninins  our  data,   we   identified    ide^  units 
with   functions   which  are   slTiilar   to   tha   function  of  a 
narrativa  clause   (cf.    Labov    1972).      Tnese   appeared  in 
loosely  coherent  groups  of  idea   units,   with  evaluation 
eusedded   within   the   Idea   units,   or   with  evaluative  co^- 
m»ntary   franing   the   identifiable  groj?  of   idea  units. 
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Tnese  groups  of  idea  units  usually  appeared   in   the   fo r-n 
of  discussion   of  a   series  of  events  or  observations, 
Tiora  or   less   in  order  of  occurrence.      In   general,  tney 
served    the   function  of  illustrating   through  the 
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considered   a   sub^-category  of  Evaluation. 

In   the   following  exanple,   the  student  evaluates 
the   topic   he   is  considering,   and   then   proceeds  to 
relate  a   series  of  events  as   an   illustration  of  his 
point: 


(25)     •  »   •  but  •  •  oh  I  knowf  oKt  I  Knott  one  that'i  good  hcrcs» 

In   the   process,   he   is  filling   in  necessary  information 
for    the   interviewer,   and   it   is  recounted   as  'a   series  of 
teTipo  rally  ordered   events.     The   effort  to   use  these 
events   to   illustrate   a   point  requires   that   the  student 
link  nis   past   experience   to   his  present   task.     Thus  a 
transition  between   two  worlds  occurs   in   idea  uni't  25.* 


(25)  •   •   •  but   •    •  ch   I  knowi  ok  i   I  kncv.  cr€  that's   socd  herest 

(26)  •    •  •   that  I   ca  .   •  I  came  here   in  the  beginning  cf  the 
year  ^ 

(27)  •    •  •   and  I   hsd  a  ca  •   •   I  had  fry  schcdulet 

(28)  •   •  we  got  tc  pick  our  schedules. 

(29)  •   •   •  *nd  so   I  had  my  schedule.  ^ 

(30)  •   •   •  And  .    •   •  the  georetry  class  I  y*as   In  ^as  tco  bJg. 

(31)  .    •  •  So  they  took  .   .  they  took  .ire  cutt 

(32)  .   •  transferred  me  out  cf  my  geometry  classt 

(33)  and  put  me   5n  with  the  sixth  perlcdi 

(3<i)       •   •   •  which  freans   •  •  which  meant   I  ha<?  to  change  my 
English  class  too.     (Dor,  Interview) 


Narrative-like  segments  were  also, embedded  witThin 
the  evaluations  of  plans  for  composing,  as  can  be  seen 
in   tne   following  example: 


(15)  .   •   •  Hew  woulc  I  go  about  writing   It.  fdaudhs) 

(16)  •   •   •  Probably   •  •   •  act  llket 

(17)  •   •   •  ufT  •   •  write   It  more  like  a  stcry.. 
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(le) 

(19) 
(2C) 
(21) 
(22) 
(23) 
(2<»  ) 


•  Like  **c're  urf*  •  •  •  Just  rushinc  cut  of  fourth  pcriodr 

•  and  •  •  ycu've  gotten  all  your  books. 
.  For  ycur  next  class. 

•  Run  down  the  halls. 

.  And  to  stand  In  a  fclg  n\}t  1ot>c  line. 
.   I'm  exaqgeratlng  a  little  bitf 
but   .  .  yoL  know  .  •  to  get  the  pc  Irt  across. 
(Katie)  " 
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Labov  defines  evaluation  3S  "that  part  of  a  narra- 
tive which  reveals  the  attitude  of  the  narrator  towards 
the  narrative  by  emphasizing  the  relative  inportance  of 
so-ne  n-arrative   units  as  compared  to   others  "  (Labov, 
1973,   p.    37).     Ha  also  distinguishes  between  degrees  of 
enbeiding  of  the  evaluation  in   the  narrative  franfjework, 
with  synbolic   acti.ons  being  the  .tiost  "internalized" 
type,   and   " d ir ec t  »t a t em  en ts  of  the  narrator   to  the 
listener   about   h  i  s  f  e  e  1  i  n  g  s  ,  "   being   the  most  "'exter- 
nal."    Idea   unit   (2^)   above,"    .    .    .    I'm  exaggerating  a 
little   bit,"   corresponds  to  Labov's  notion  of  "exter- 
nal" evaluation. 

Evaluative  3t£ategijes.      Besides   the  overtly 
e  V  ai  ua  tlv  e'coTTTsen  t  s  such  as  the  one   in   idea  unit  (23) 
above,   we   found   that   students   adopted    a  variety  of 
evaluative  " s t r a t e g ie s . "   The   following   types  were  pre- 
valent  in  our  data: 

1.  Attribution 

2.  Generalized  Attribution 

3.  Generalization 

U.  Comparison 

Tnese  strategies  appeared   in  both  the  oral   and  the 
written  data.      Several   differences  between   the  oral  and 
the  written  data  were  revealed   in  our   examination  and 
will   be  discussed  below.      Briefly,   however,  they 
Include  a  difference   in   audience,  a  difference  in 
sltuationt   and   a  difference  between  oral   and  written 
language. c per   se.     These  differences  complicated  our 
task  of  cross-referencing  between  the   two  sets  of  data. 
However,    in   terms  of  content,    related   groups  of  idea 
units  were  readily  Identifiable, 

^       Attribution.     Speakers  and  writers  make  judg- 
ments in   tne  development  of  an   argument.     As  language 
users,   our  general  d i s po s I t io n   i s  to  categorize  ani  to 
characterize  according   to  experience.     Any  event  has 
the  possibility  of  being  represented    in  different  ways 
through  the  assignment  of  different  roles  to   the  people 
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involved*     Tnis  prooess  operates  at  several  levels. 
For  sxa-nple,   at  one  level,   we   can  assign  the  roles  of 
antagonist/protagonist.     At   another   lev^l  ,   we  can 
assign   se-nantic- syntactic  roles,   whiah  forn  contrasting 
pairs  such  as   a  g  en  t/ pa  ti  e  n  t .      In   so-^e  cas'es,  assigmient 
of  roles   fs  purposeful   a'li   serves   an   evaluative  func- 
tion. 

In   the  process  of  examining  our  data,   we  found 
that  many  of  the  students  attributed   opinions  and/or 
experience  to   others.     In  assigning  opinions  or  experi- 
ence to   oth-fers  while  recounting  events,   the  student 
provides   additional   support  for   his  point  of  view,  thus 
"bolstering"  his  argument.     Generally  speaKing,  tfte 
students'    assignment  of  roles   to   individuals   and  groups 
appaared^  to   have  an  evaluative  function. 

In   our  data,   when  students   assigned  opinion  or  the 
role  of  " expe r i encer "   to   someone  other   than  themselves, 
the  result  was   somewhat  similar  to  what  Labov  has 
described  as  "embedded"   evaluation.      In  discussing 
evaluation  in   narrative,    Labov   cites   two   examples  of 
evaluation  that  are  deeply  embedded.      One  example 
involves   a  narrator   addressing  a  second   person  within 
narrati  vd'-inter  nal   time   (Labov,  ,   pp.    372  ). 

Corresponding   to   this  case,   some  students  in   our  data 
Pefgrenced  a   second   person  to   indicate  that  their  opin- 
ions were  not   exclusively  their  own.      This  device 
creates   a   consensus  group  to   support   their  arguments: 


(^9-51)     Causf  rry  friend  he  always  says   •   •    •  It's 

a  pain.     Yet  nevfr  get  what  ycu  wart  or  anythina. 
( Ch  I  p  t  1 nt e  rv I ew ) 
«  «  « 


(18)  *  friend  and  1  w^re  dTscussing  another  way  of 
solving  the  litter  prcLlcrP 

(19)  and  we  came  up  v%ith  this  sugcesticr. 
(Carif  Interview) 


«  »  t 


(13)  A  lot  easier  for  me  and  rr.y  frlerdst 

(1^)  And  .   .  well.  I   guess  my  friends  are  annoyed  toe, 

(Cent  Interview) 


»  ff  t 


Labov's   second   j^xample  shows   the   introduction  of  a 
third  person  making  an  c v al ua t i v e  comm en t ,   which  has 
th^e  effect  of  increasing  dramatic   force  because  the 
comment    i^'omes    from  a  "neutral'*   observer.     One  student 
in  ourdat^  wrote: 


•i  0 1, 
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fl55    One  other  r»4Son  !  chose  this  subject 

was  t>rcsuse  brother  »!sd  h*d  prDblems 
v,#Jth  the  opcn  classroom. 


in   several   in  5 1  an  c  e  3  .   5 1     e  n  t  s  provided  ^'I'l'll^l'll' 
port  for   their  arguments  by  r.efarenc  ng  ".Jt  only  th  ir 
ftUT  exD-rience.  but  also  that  relatad   to   then  by  rela- 
tr  -.ro;   friends.     Thus,  while  Labov's  --P/"/ 
t-ate  evaluation  eiibedded   in   the  coTinients  of  the  ^^^^^ 
tor   to  Character,   (or   the  comments  of  characters  to  the 
narrator)    in   n  ar  ra  t  i  v  e- in  ter  n  a  1   J  i'"  «  '   °  IJ''  J  ^  ^ 

revealed  embedded   evaluation  in   the  form  of  shared 
opinions  and   "sharevi"  experience. 

Generalised  Attribution.     In  examining  ^l^l/' 
also   fT^Iirriri^rrin  stances  of  embedded  e  v  al  ua  1 1  o  n  wh  er  e 
wnole   idea   units  were  usedto   introduce  larger  con- 
sensus  groups.  have  named   this   strategy  "general- 
ized attribution"    to  distinguisn  it   from  the  strategy 
of  attribution  where  consensus  groups  are  limited  to 
specific   individuals   (or   groups  of  individuals).  Tnese 
|?~H?Ie3  ware  signaled  most  clearly  by  mass  o/  P^^^^^ 
nouns  (kids,   students,   people),   plural   pronouns  they 
and  ^      indefinite  pronouns   (everybody,    e v er yon e ) .   an d 
th'  generic   pronoun   (you),   which  generalizes  the  attri- 
h^tlan   to  tnc^ude   anyone  confronted  witn  the  situation 

rdescJibed        GaLralized   attribution  can  attribute 
either   negative  or  positive   characteristics  to  large 
Irou"?.   ^^aring  "consensus."      In   the   following  example. 

'^^'^  'iVUl/mi^^^^  lySonii^ul  SJCrSSa?  group 
?rwnlch%ne'att^!rutes  an  authoritative,  decision- 

ma  <i  n  g  ro 1 e  : 


da) 


People  have  tried  a  lot  of  ways  — 


f  «  • 


(ipl       We've  tried  a  lot  cf  things  —  and  .   .  nothing  seems 
tc%^t  through  their  heads.     (Carla.  Interviev.) 

la  another   example.  Martin  attributes  to  "students 
i.re"    an   "attitude   problem"   which,   he   claims.   Is  at  the 
root  of  all   the   problems  he  learned  about  in  connection 
with  planned   improvements  at  the  school. 

(13)       .   .like  the  *aY  thev'rf  'bove  evfrylhing  else. 
(Hart  In,  Interview) 


438 


ScjJents  sea.-n   to   establish   negative  attribution  gnoups 
for  the  purpose  of  aligning   tne-nsalves   in  opposition  to 
these  groups.      To   establisn   support   in   their  opposi- 
tion,   they  may  create   positive  attribution  groups. 


(21)     It  wDuTd  ttork  by  student  Ignoring  other  student 

who  dDn't  care  and  telUn9  thei  that  they  dj>  care 
what  happens  to  the?r  school*  (Kartint  essay^ 


Wnile   the   assigniisnt  of  roles  contributes  to  "con- 
trasting"  evaluation,    sone  cases  of  generalized  attri- 
bution  achieve   tne  reverse   effect^  the   impression  of 
universal   agreenent.      Generic   p'-onouns  "you",  "one", 
"everyone",    "everybody"   are   abstractions,   markers  of 
the  speaker/writer  assuming  that,   within   the  particular 
conditions  being  described   in  the  generic   present,  all 
people  would   react  in   the  same  way.     The  generic  pro- 
noun is  often   accompanied   by  a   conditional  predicate 
frame  or  some  other  comparative  context,    e.  g.,  negative, 
future,   question,    (c.f.   discussion  on  comparative 
evaluation),   as   in   the   following  exa-nples; 


•  •   •  and  then  •  •  you  sort  of  .  •  lose  friends.  (Joe) 

«  «  « 

(16-19)     •  •  At  hojte  you  »<ou1dn't  have  people/.   •  •  you  kno* 

•  •   •  throwing  things  on  the  ground  and-— »  (Carla) 

«  «  • 

(6-75     When  you  ars  distracted  fronn  the  c1  as  sr  ooti, /you  win 
end  up  (n  trouc>Te/(6)   and  your  grades  mMI  drop 
drastica It  y* 


Related   to  both  attribution   and   the   following  dis- 
cussion of  generalization   is   the   use  of  an  inclusive, 
generalized   attribution   to   "ev<iryone"   and  ''everybody'^ 
which  functions  to  expand   an  established   reference   to  a 
consensus  group   including   the  narrator:   "we",  "every- 
one"  and   "everybody"   mark  the  consensus  group   "we"  as 
all-inclusive,   or   without  exception.     "Everyone"  and 
"everyoody"    in   the   following   exampleCs)    are  accompanied 
with  the  generic   present  tense. 


<15)       •   •  whereas  everybody  •    .  tytt.ibQC!y   •t  Central 
(16)       ...   1  frean  cvcrytody  v.ant5   .   .   tc  C2c  It  the  way  we^ve 
been  doing  lt»  (Alicet  Interview) 

i  «  ff 

(62)       •   •  •  Fecause  that  seems  to  give  the  ffcst  problem  to 
everybody.   (J«y»  Interview) 
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Anotner  variation  of 
the   introJuction  of  a  hypo 
rep res  an  ting   all  students 
cu-n  St  a  no  as  * 


this  general   type  involves 
thetical  "individual'' 
within   the  saiie  set  of  cir- 


i7l\     Bv  chanqina  a  StulCDlli  c7»ss  «nd  teacher  It  first  sai^es 
riethods.  I0on»  esssy) 

(17)     Gn:  .»ould  think  that  i.iLulCQt  would  be   interested  In 
ho.*  his  icnosl  functions,  (Kim.  essay) 


In   this  last  exa-nple.   this  "hypothetical  individual" 
beca-ne  the  main  strategic  vehicle  adopted  by  the  stu- 
dent in   developing   her  argument. 

Generalization.     Generalization  forms  a  third 
type  of  evaluative  strategy  which  is  characterized  by 
•nodification  of  a   category  to   widen   its  natural  boun- 
daries.     This  alteration  allows   the  sp e a Ke r / wr i t e r  to 
avoid   direct   statement  or   specification,   and   thus  to 
accomnodate   the   audience's  capacity  for  assimilation 
and   agreement  with  conditions  specified   by  the 
spi3Ker/wri tar  .     Generalization,   when  applied   to  sub- 
Jects'or  objects,   appears  in  the   form  of:  dummy 
subjects/objects,  mass  nouns  and  pluralized   sets  in  a 
generalizing   ( no n a t t r i b u t i v e )  context. 
Carla  uses   the, dummy  subject  in  her  essay: 


(3)  11  Is  really  Ircredib>e  tow  Us  so  difficult  to  pick 
up  a  piece  of  paper  and  put  il  Intc  the  garbage  can. 
(Carla.  Essay) 


and  mass  nouns  in  her  pr e i n t e r v i e w : 

(30)       ...  a  let  of  people  think,   (Carla,  Intervie*.) 

Martin   uses  3   definite  article  with  a  favored  mass 

noun: 


(<i7)       I    t^inl'  tht-BSCElS  •    •  Lrlted  States, 

(*P)       .    .  t^e  peoplf  in  the  school   are  tht  rest, 
(Martin,  Intervlet») 
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'15,; 


finen   applieJ   to   actions,   generalization  transfor-ns 
actions   into   conditions  or   states  characterizing  ine 
sjDject,   appearing   in   the   for-n  of  tne  generic  present 
and    tne   progressive  participial.      Progressives  serve  to 
describe   the   action  of  tne  -rjodified    verb   as  constant 
andongoing. 


(lOJ  Lfice  all  these  people  •  •  •  itaadiEU  1"  front  of  we 
(11)    9l££CCiUClIIafl  something  (Joe) 

«  «  « 

121 )     ATI   the  people  aLS-C32iQa  here 

(22)     •  •  «nd  tLSiiJlillfl  other  schools  up*  (Don) 


Finally,    "hedges'^   occur   within   the   student  data,  serv- 
ing   to   generally  obscure   the  boundaries  of  a  category 
by   expanding   ("a   lot  of")   or   constricting  ("so.-ne/* 
":nany,"    "sort  of/and   "kind   of")    to    fit  the   purposes  of 
the   sp=»aKer/   writer   at  the   point  o^  his  discussion. 

Obviously,   more   features  belong   to   our  category 
"generalization"   as   we  have  defined    it.      ^e   have  rei- 
strictei   our   analysis   and   discussion   to   the  io s t   r e a d i  1  y 
identifiable   exanples  found   in    the  data.      Before  ^ 
directing  our   attention   to   tne  next   evaluative  category 
in   our   discussion,   we   should   note  here   that  generaliz- 
ing  modifiers   appear   in   "clusters,"    and   often  serve  to 
"qualify"   evaluated   Tiaterial,   such   as  in   the  following 
ex  a-Tip  I  es  : 

Joe   evaluates  his  identification  of  a   problem  with  one 
.Tiass  noun,   one   pluralized   noun   and   a  generic  present 
tense  fom  of  "to  change": 


(16-17)    .   .   ard  the  way  •   •     Btcalt  Tntc 


Martin  evaluates  his  probleTi  by  using  the  "h*»d2e"  "kind 
of "  :  " 


(12)       I   think  the   •   .  student?  here  have  s   •  •  l^lod  Qf  a 
attitude  problem.   (Martin^  Interview) 


Jay,  balancing  the  pros  and  cons  of  his  choice  of  prob. 
leTi,   uses   two   hedges   and    tne   generic   present  tense: 
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{!%) 


«15) 
(17) 


3ut  then  •  •  you  biiC  »o  "Jch  nols^  •round* 


•  •  that  It  ilnl^af        •  •  disturbs  your  uh  •  •  thinning. 

•  •  ^u^iuCUla  really  get  that  •  •  cooped  up  feeling 
all   over  t  (Jay) 


Tnsss  instances  of  g e n a r a L i z 3 t i o n  serve  the  function  of 
qualifying  evaluative  conuentary. 

Compa  r  ators.     Labov's  category  of  evaluative  "coti- 
parators'*    has   tieen   adopted,   intact,    in  our   analysis  and 
discussion.     According   to   Labov,   comparators  include 
the   following   forns:   negativps.    riitn»"es,  q'J3sticns, 
or-clauses,   superlatives,   and   co.upa  r  a  t  i  v  e  3  .  Introduc- 
ing  his  discussion  of  comparatives  with  negative  forifis, 
Labov   describes   these  as   representing   the  defeat  of  an 
expectation  "that  sonathing  would   happan''.  "Nagative 
sentences  draw  upon   a   cognitive  background  considerably 
ricner   than   the   set  of  events  which  ware  observed. 
They  provide  a   way  of  evaluating  events  by  placing  than 
against   the  background   of  other   events  which  ^ight  have 
happaned,   but   which  did   not  occur." 


ExanpLes  in  our  data  indicate  tnat  this  is  indead 
the  casa,  espacially  when  the  negative  is  used  in  coti- 
bination   with   tne  conditional  aspact: 
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(31^32)       •   •    •  you  know   .   .  prople   sit  here»  »   •    >  and  there 
•   t   .  you  can't  sit  with  th*riri   (Jcr?  Intcrvle^i) 
I  ft  » 

(6)  Unf  c  rtuna  t  cl  y  I  r>any  students  and  teachers  don't 

feel  as  fortunate  as  I  do.   (Joe»  Essay) 

am 

(^}  •   .  because  urn  .   .   .  litter  seerrs  tc  be  a  big 

problem  when  ft  really  shculdn't  be. 
( C  ar 1  a  »  I  n  terv  i  ) 

ft  I  « 

(1^)  ...  At  hcffe  you  wouldn't  have  pecple* 

.   .   .  you  know  *   .   .  throwing  thircs  on  the  ground 
and — t   (Carla»  Interview) 

ft  ft  ft 

and  kids   just  dcn't  show  up.   (Ki-r,  Interview) 
ft  ft  » 

(30)  •   .    it  wasn't  landscaped  theni 

(31)  .    .  and  every  time   I've  been  at  schcol    It's  never  been 
landscaped.     (Sobi  Interview) 

ft  ft  ft 


.    •    .    Hy  Q 

(Dor,  Inte 


rades  haver't  been  tco  velli 
rv  i  ) 
#  «  i» 


They  won't 


let  us  dc   It*    (AHcCf  Irtervlew) 


The  following  is  a  particularly  plaintive  case  of 
einbedding  of  defeated  expectation: 


Wnile   generalization  usually   servas   to   expand  a 
given   category,   conparing  and   contrasting  through 
modality  essentially  serves   to   apeci  f y   the  conditions 
under   which  a   category   is  presented.      This  is  espe- 
cially  true   when   futures  and   conditionals  are  associ- 
ated  with  evaluatively  marked   co.Timen  tary .  Fillmore's 
description  of  modality  as   it   functions  in  case  gram-nar 
provides   us   with  the  clearest   delineation  of  modal 
function:    "modaLs   serve   to  overtly  mark  an  inference 
that   the   spea^or  is  making   .    .    .   when  a   speaker  shifcs 
mood,   he   indicates   something   is  likely   (will),  possible 
(may),   desirable   (should),   obligatory  Xmust)  " 
(Fillmore,    1930).      It   follows    that  conditional  modals, 
would    and   could,   imply  conditional   likelihood   and  pos- 
sibility respectively.      In   investigating   our   data,  we 
found   that  the   use  of  conditional  modals   in  both  the 
preinterviews   and   the   essays   was   pervasive.     Within  the 
pre  in t er V ie ws ,   conditional  modals  were  largely  based  in 
the  hypothetical  nature   of  the   task  in   which  the  stu- 
dents ware  engaged   (describing  what  they  would  write). 
Typical   examples  include   tne  following: 


C7)       •  •   •  And  •   •  this  Is  probibly  «#hat  I'd  write  about   •  • 
because-"^ 

(6)       ...  irfcn  one  of  the  things  I  ^^oul  d  writs  about*  (CarU) 

«  f  < 

(^0)     «  .    .  And  .   •  that  would  probably  be  the  um  •   •  second 
par  agr ap  hy 

(95)     •  •   And  that's  probably  how  I'd  .   .   close  off.  (Kir) 


tl6) 


I  have  firmly  believed  that  iiany  teachers 
helping  the  studsnts.  (Joe,  Essay) 


don't  1  ike 
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In  otner  cases,   conjitionals   s^rve  to  co n d i t i o n a  I i ze 
tie   soljtion  of  a  p-'obleTi  to   the  student's  suggestion 
Xn2ss   are  often   framed   witnin  a  ogjsal   if-then  con- 
struction : 


ill)       I  think   If  wc  had  an  enclosed  cafeteriat   (C»rl»f  8$$''/) 

«  ft  ff 

tfi')       •   .  •   If  •   .   ycu  know  •    .  kc  could  pet  rrore^-  •  • 
student  participation. 

•  •  we  could  Probably  .  -  solve  most  cf  these  problerr^— - 
(Carl  a  t  Essay) 

tt  ft  ft 


(7)       bfcause   if  the  student  had  a  better  acditude 

twerd  their  school 
is)       It  would  be  •  better  school. 

{^)       If  the  student  gave  so  respect  to  the  school 
CIC)       It  would  do  rpcre  than  anything  else  cctld* 
(Partlnt  Essa>) 


«  ft  ft 


{2?)       T^en   if  you  dc  this 

C23)       you  would  improve  the  school  more  thcr  landscaping 
more   than  C.I. P.   (Martirt  Essay) 
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Tnese   if-then   and  coTiparable   (when-then)    co  n  s  t  r  u   ti  o  n  s 
appeared   in   several   essays.      Perhaps   the  nost  interest- 
ing   cases  of  modal   usage  involved   if-then   franing  dev- 
ices qualifying   the  proposed   institution  of  the 
student's  suggestion: 


(2a)       The  sucgestlcr  that  I  would  rake 
(3)         If  I  wis  askec 

{2b)       Would  be  the   attitudes  cf  people  at  Central.  (Joc,f$5ay) 

«  ft  ft 

(H)       The  basic  goal  that  I  would  have   Is  trying  to  work 
tcqether  and  get  everything  pulled  together* 
(Jce»  Essay) 


Rnetorical   questions   also   fit  a  discussion  of  evalua- 
tion and   are  also  representeJ   as  a  rather   so pn i s t ic a t e J 
evaluative  device  produced  by   Kiii   in   her   essay.  In 
rhetorical   questions,    the  answer   is  "planted"  within 
tne   fortj   of  the   question   and    the  question   is  addressed 
directly  to   the  audience  with  the  assuTiption  of  agree- 
ment, being  a   function   of  the   fornn.     Associated  with  her 
use  of  tne  device  is  the  conditional   frane  signaled  by 
"if*»,   and   the  hypotnetlcal   student  established   in  pre- 
vious S3ntance3: 
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(26)  If  a  student  dorsn't  bother  to  take  pride  in  his  schooU 
knoi^  hop^  ft  functionsr  and  hold  interest  in  activities* 

(27)  howt  do        knoirf  he  will  develop  an  Interest  later  on? 
(23)     wni  he  picic  up  litter  fn  his  officct 

(29)     or  win   his  offfce  resenbls  a  plgstyt  reflecting  on 
his  work? 

(3D)     Will  he  register  to  vote  for  the  president  of  the 
United  States 

(31)     or  will   he  choose  to  let  other  people  elect  the  president 
for  hJTi?  (Kim) 


Cooipar  ison   of  Oral   and   Written  Units 

An   underlying  assu.Tiption  in  our   analysis  was  that 
the  student   is   accessing   in  f  o  r  ai  a  t  i  o  n   in   the   process  of 
reading   tne   pro.Tipt   wnicn  he  can   then   use   in  composing  a 
written  response.      Tne   construction  of  an  interpreta- 
tion of  the   pronpt   is   tnus  viewed   as   an   integral  part 
of  the  coTiposing   process.      In  order   to   determine  how 
students  used   their   initial   interpretations   in  their 
written  c  ooi  po  s  i  t  i  o  n  s   we   conpared   segijents  of  the  oral 
interviews   to   segTients  of  the  written   conpo  s  i  t  i  o  ns  . 
Both  sioiilarit   3s   and   differences  were   found  between 
the  oral   language  of  the   interviews,   and   the  written 
language  of  tne   students'    coti  po  s  i  t  i  o  n  s  .     Basically  the 
si-nilarities   were   found    in   content  and    tne  differences 
were   found    in   rnetoricai  function. 

Differences   between   oral   and   written   language  have 
received  Tiuch  attention   in   the   past   decade  by  linguists 
worKing   in    the   field   of  text  analysis   (Ochs,  1979, 
Cnafe,    1979,   Tannen,    1973).      Mjch  of  this  work  has 
focussed  on   the   ways   in   wnich  the   sane   ideas,  concepts 
or  events,    are  expressed   differently  in  written  and 
spoken   language,    with  contrasting  descriptions  of  the 
characteristic^i  of  the   language  used.      Tannen  has 
noted,    for   example,    that   "...planned   written  discourse 
makes   use   of  complex   syntactic   structures,  formal 
cohesive  devices,   and   topic   sentences.      In   general  it 
is  more  compact  "    (Tannen,    1973).     Chafe  has  also  com- 
mented  on   this   characteristic   of  written   language,  in 
describing    it   as   "integrated."   Spoken   language   in  con^ 
trast,    is   often   loosely  strung   together   with  coordinat- 
ing  conjunctions   (or   with  no  overt  marking   of  th-  rela- 
tionships  between   propositions   whatsoever)  (Cnaf- 
1979).      Tne   result   is,    that   ideas,   descriptions  of 
characters,    and/or   events   which  are   typically  expressed 
in   several    loosely   joined   clauses   in  oral   lan^uaae  ar- 
often   expressed    in    fewer   words   in   written  lan^ua^e 
because   of   the   greater   degree   of  integration   on-  is 
able   to   achieve   in   writing   through  processes  of^ubor- 
dination   and   subject  deletion. 
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In   Dur   data,  differences  b  et^e^n   oral   an  i  written 
units  reflecting  connon  content  were  Dotn  a   function  of 
tne   t  r  an  sf  om  3 1  i  o  n  s  wnich  occur   betwsen  oral    and  writ- 
ten  styles   and   a   function   of  the  necessity  to  perfom 
witnin   two  diffsrent  situations.     One  situation 
involved    the   student   interacting   with   bota   the  prcnpt 
and   an   interviewer.      The  other   situation  involved  writ- 
ing  an    essay  in   response   to   the   proupt   wnicn   would  be 
evaluated   by   tne  teachers  at   the  school. 

An   exa-nple  of  the   sort  of  conpression  that 
occurred   in  our   data   follows.      Tne   student's   oral  con- 
ments   are  his   initial   responses  upon   reading  the 
proTipt,      Tne   first  idea   unit  rnaKes  it   clear    that  the 
stjjent   is  referring   to    the   first   sentence  of  the 
prompt.      Later,   he   refers  to   tne   senond   sentence  (idea 
uni      n  )  : 


12)       •   .   .   Ah,  .   .   the  first  thine  abcut   ?r  provcrr^erts  i 
(?)       Y'know  like,   that  I  likei  ah    .   .  lardscapfngi 
(^O       they  are  thinking  about   .   .  un 
C5)       •   .    •  giving  nrcney  for    •    ■  to  rrake  tht  school 

1 cck  a  little  better . 
(C)       And  the-- 

(7)       .   •    .   they've   .   *  they've  qot  a  bic  crarti 
(&)       •   ♦   .   state  or  federal   or  sorrethinc, 
(9)       •   .   .   tc  improve  the  curriculum 

(10)  ..•of  the  s  chool • 

(11)  .   ,   •   I  knowi   the  thing  about  a   .    .  siti-aticn 
or    .   .  pr cb  1  e rr  f 

(12)  I   think   the  .    .  students  here  have  a    .    .  kind 
of  an  attitude  problem.    (Hartin,  Interview) 


In  Martin's  written  composition,  the  "sane''  topics  are 
cohered   in  his  opening  sentence: 


CD       Jht  rrany  suogcstlon  for   Improverrent  have  ^een 
frade  landscaplngi   the  C.I. P.   Grant,  and  the 
•ddlteud  (sic)   of  the  students.   (Martlnt  Written) 

f/nile  a  total  of  72   words  are  devoted   to    providing  a 
background   to    the   introduction  of  these   ideas  in  the 
or3il   interview,   only    18  are   used   in   for.ning   a  list  in 
tne  written  c  o  n  po  ai  t  i  o  n  •     The   co-npression  that  occurs 
in   these   exanples  is   partly  due   to  differences  in  the 
characteristics  of  oral   and   written  language.  However, 
coTipression   also   occurs  because   tne  audiences  of  the 
oral    interview   and   the  written   composition  are  dif- 
ferent.     In   the  oral   interview,    the   co.Tments  are 
ajJressaj    to   an   outsider,   a   "naive"    interlocuter,  who 
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listing   references   to   the   first   sentence  of   the  proupt, 
Martin   demonstrates  his  awareness  of  the  "many  sugges- 
tions".     By   framing  his  own   subject  within   the  l*i  s  t  , 
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Strategies   adopted   by  the   students  clearly  .shifted 
as   a   function  of  how  they  regarded   their   task  and  oral 
versions  of   ideas   that   occurred    to   the   students  in  the 
pi-ocess  of  reading   and    interpreting   the   prompt  were  not 
always    longer    than   tneir   written  counterparts.  Oral 
versions  of   ideas  were  often   expanded   in   their  written 
counterparts-      In   contrast  to   Martin,    Katie   never  con- 
textualized    the   first   sentence   with  background  informa- 
tion.     Sne   identified   lunch  lines  as   a   problem  at  the 
school    in   her   initial   response,   and   then  made   the  fol- 
lowing  comments   in  describing   how  she  would   "go  about" 
writing   ner  paper. 


.    •  liV^  we're  urr  .    .    .   Just  rushlnc  CLt 
of  four th  pr r  led  • 
(jc,)       .    .   .  and   .   .  you'wr  qcttcr   all  ycur  tccks. 


•  "  Con  t  ex  t  J  a  I  1  za  t  i  0  n"  refers  to  tne  p»*oces3  whare:^^  a 
reaJer  (or  speaker)  creates  an  imagined  context  for  a 
particular   u'  terance  in   coii prehension. 


ERIC 


447 


U' 
4  r ;  '  » 


(2C) 
(21) 
(22) 
(23) 
(2A) 
(25) 
(2t ) 
(27) 
(26) 
(29) 

(30) 

(31) 

(32) 


n  e   Icng   1  I  re  • 
p  oint  across. 


•   For  your  next  class. 
.   Pun  down  the  halls. 
.   And  .   .   to  stand   In  a  blg^ 
I'r>  exaggerating  a  little  bltt 
but   »    •  yciJ  knc>^   •    •   to  get  the 
.   That's  hew  I'd  wr  ite   It . 
And  then   .   .  standing  therci 
for  half   the  •   .   lurch  period* 
wait  for   you  lunch. 
,    (laughs)  Af>d  then  ycu  finally 
outside  and  eat  it. 
.   .  .  And  then  .   .  the  perlod'd  be  ever 
stni   eating  ycur  lunch. 

!   .  1   5  .   .   .  And  then   ...  ycu  cculc  finish  ycur 
lunch  and  then  go  to  class--#  -  ♦ 

.   .  like  P.E.   (lauchs)    .   .   .   It's  net  nee  tc  go 
to  class  when  you've  just  hurried  up  and  eaten  your 
lunch.     (KatiCf  Interview) 


cet 


go 


your  lunch 
while  you're 


In  ner  w-itten   version,    Katie   devoted  many  .nore  words 
to  covering  tna   sane  material:   -^nat   is  originally 
treated    in    14   idea  units   In   the  o^al    interview  is 
expanded   to   25   segments   in   tne  written. 


1  Bu zzzz  • 

2  The  five  ulnute  bcTl  ranc. 

3  My  fourth  period  teacher  had  finished  his  lecture 
^  and  we  had  the  rest  Df  the  period  for  ourselves. _ 

5  I   scrawlpd  out  one  last  matn  probleii  before  closing 
my  books. 

6  "Ho^  «uch  tl«e,  KIrsten?" 

7  1  asked 

B  "Two  minutes.* 

9  Kirstsn  said  after  looking  at  her  •^atch. 

10  X  put  my  homework  paper   into  «y  bindert 

11  and  stacked  my  books   In  my  ar-ns. 

12  BZ22« 

13  1   jumped  up 

m  and  raced  out  of  the  roon  fitrglng  into  the  traffic 

of  other  students  hurrying  to  lunch* 

15  I  ran  to  my  lockicr» 

16  Shoved   in  ry  books 

17  and  sprinted  to  the  cafeteria. 

18  There   I  -fas  pet  by  »  gigantic  llnit. 

19  My  stomach  growling  1  waited  In  line  for 
f I f  teen  «! nutcs • 

20  k^ith  •  sigh  of  relief  I  jot  my  lunch 

21  and  went  out  side  (sic)   to  cat. 

22  Half  p^ay  (sic)  tnrough  ity  lunch  tne  belt  rang 
signal  in?  (sic)  for  the  end  of  lunch. 

23  This  time  my  sto«nach  didnt  (sic)  growl  t 
2M  I  did! 

25  This  situation   Is  a  slightly  exaggerated  ,       -       rr  v/i 

exan^lc  of  one  of  the  ««ln  problems  at  Central   High,(Katic»  EssaY' 
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tnat   taKes   place   sonetine   in   the   past,    tne   autihor  takes 
tne   role  of  protagonist  and   the  narrative  proceeds  fron 
her   perspective.     An   additional   character   is  introduced 
as   well   as  dialogue.      Most  of  the  expansion  occurs  in 
lines    1   throagn   12   in   the  written   te;{t»   where   Katie  is 


i  n  t  r 

0 

duel 

n? 

n 

e 

w 

el  en 2 

n  t 

S    0  f 

detail    in  creating 

a  con- 

tex  t 

for 

the 

e 

pi 

sod  e 

sn 

e  wi 

snes   to   present.  Al 

though 

diff 

j> 

rent 

r  n 

/> 

t 

0  r 

i  c  a  1 

St 

rate 

gies  are  adopted,  th 

e  under 

1  yi  n 

St  r 

uc  t 

u 

r 

e 

of  th 

e 

even 

t   is  the  same   in  the 

oral 

and 

w 

r  i  1 1 

e  n 

s 

a 

en  t  s  . 

Tne 

"sane^*   event   sequ  enc 

e  s  o  c  c  l: 

betw 

en  1 

i  n  e 

s 

13 

thro 

og 

h  13 

in   her   written  text 

,  and 

betw 

en  1 

i  n  e 

s 

18 

tnr  o 

ug 

h  22 

in   the  interview. 

Co  71  pa  r  - 

abl  e 

sequ 

enc 

e 

s 

a 

1  so  o 

cc 

ur  b 

etween   lines    19  thro 

ugh  22 

in  h 

r  wr 

i  1 1 

n 

t 

e  X  t  a 

n  d 

lin 

es   26   through   30  in 

the 

in  t e 

r 

view 

Katie 

i 

3 

the  5 

ingle  ex  an pie 

in  th 

e 

subsample  of 

e  1  e 

V  e  n  stud 

a 

n 

t 

s  of 

a  c 

onpe  t e  n  t  writer. 

wno   is  aware 

of 

tne 

p  0  s  i  t  i  V 

a 

e 

f  f  ec  t 

s  a 

vividly  e 

X  pr  ess 

ed 

personal  " 

con- 

tex 

tualizat 

i 

o 

n 

"  can 

ha 

V  e  on  the 

point 

o  n 

e  is  trying 

to 

m  a  k 

e.  Sne 

b 

e 

Z 

ins  h 

er 

corrspo  si  tio 

n  by  e 

nb 

arking  inne 

d  i- 

ate 

1 y   on  an 

e 

X 

pe  r  i  e 

nee 

rooted  in 

a  per 

so 

nal  perspec 

ti  V 

e ,   con  pi 

e 

t 

A 

with 

so 

un  d  s  »   a  c  ti 

o  n  s  ,  a 

nd 

feelings 

d  e  s 

c  r  i  b  ed  i 

n 

3 

2q  J  en 

c  e  . 

Once  thi 

3   CO  n  t 

ex 

tualization 

is 

con 

p 1 e  t ed  , 

i 

t 

test  i 

fie 

s   for  her 

a  r  gune 

n  t 

Our  p'j 

r 

P 

0 

s  e  in 

pr 

esenting  h 

er  o r a 

1 

and   wr i  tten 

d  at 

a    is  not 

t 

O 

disc 

us  s 

the  ner  i t 

sofa 

3 

uccessfully 

con 

textual i 

z 

d 

ar  gun  en 

t  ,   but  to 

d  e  n  0  n  s 

tV 

ate  how  one 

St  u 

dent,    w  i 

t  h 

0 

u  t   r  i 

f  er 

encing  the 

first 

3 

entence  of 

Che 

pro 

np t  ,  nor 

t  h 

e  pre 

ced 

ent  oT  arg 

u  n  e  n  1 3 

a 

nd  suggesti 

o  n  s 

it 

presuppo 

s 

s 

.  pro 

c  e  e 

d  ed   to  con 

struct 

a 

successful 

COT 

po  si  t io  n 

Katie 

c  r 

eates   a  si 

n  g  1  e  i 

n  3 

gined  world 

wi  t 

n   a   se q J 

n 

e  of 

in  a 

ges  f ron  h 

e  r  own 

e 

xperience . 

Sne 

449 


pl3 

OS   ner  prese 

n  1 3  t  i  0 

n 

in   t  e 

interview 

f 

and   sticKs  to 

her 

plan  in  her 

COTipO 

si 

tion  , 

expanding 

he 

r  initial 

i-je 

as   to  deepen 

and  e 

nh 

ance  h 

er   nar r at i 

ve 

Sne  effec- 

ti  V 

ely  "ignores 

tri  e 

f  i 

r  s  t  s  e 

ntence  of 

th 

e   proTipt,  and 

con 

seqjently  is 

able 

to 

argue 

froTi  the 

strength  of  her 

o^n 

perspective 

Tne 

r 

e  sul  t 

iM  that  he 

r 

ess  ay  gives 

tne 

inpr  ession 

of  m a i 

n  t 

a  i  n  *  n  g 

a  consi  s t 

en 

t   v-oice  with  a 

sir 

o'ng  argUTient 

• 

Martin,    in   contrast   (See   Martin's   interview  above), 
provides   an  exanple  of  a   student  who   feels   it  is  neces- 
sary to   justify  his  choice  of  a   problem   relative  to 
wnat  he   knows    to  be   a   defined   set  of  suggestions  that 
had  been   nade   for  school   im  pr  o  v  e  en  t .      Tnis  is 
reflected   in    the   fact   that  he   lists   those  suggestions 
in   the   beginning  of  his   essay,    and   compares   his  ch:>lce 
of  subject  !natter   to   the   other   suggestions   at  several 
po  ints  in  his  essay : 


(9)  If  the  student  gave  so  (sic)  respect  tc  the  school 
(10)       It  v*ou1d  do  mere  than  anvthlnc  else  cctld. 

«    t  t 

(12)  By  bettering  their  out  1 cok   Csic)   cn  ttis  school 
It  would  Imprcve  it. 

(13)  Improve   It  n^ore  than  C.I.P» 

fit 

(It)       And  what   Is  lardscaplno 

(17)       Just  a  superf  leal    (sic)  cover  that  wculcT  te 
destroy   (sic)    by  student   (sic)  vilth  tad 
addltude.  (sic) 

t    »  » 

{22)       Then   If  you  do  this 

(23)       you  would  fn^prcve  the  school   rore  th«r  landscapsrq 

mcr e  then  C«I«P» 
(2<i)       These  other  thing   (sic)  ecu!  d  only  be  ftlly 

apr<?acated   (sic)  by  stud^^nt  (sic)  whc  C2re  about 

their  school . C Mar t I n t  6$$ay) 


DI3:U3SI3M 

The   exanpLes  above   illustrate   the  coTiplexities  we 
encountered   in   the  data.      Ideas  which  occurred   to  the 
students  could   be   traced   to   segments  of  the  written 
CO Ti positions,   but  they  varied   in   length  from  one  set  of 
texts   to    the  other,    they   varied    in   sequence,  they 
varied   in   terms  of  the  characteristics  of  the  language 
used,   and    they  varied   in   terms  of  the   functions  they 
served    in   relation  to   the   tas^  at  hand. 

A  similar   variability  in   tne  ways   subjects  divide 
the  "same'*  objective   information  is  recounted  in 
Chafe's   discussion  of  the  deployiaent  of  consciousness 
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in   narrative    (  1  933,'  PP-    1-50),      Altnougn  nDst   of  ou'- 
d3t3  was  not  narrative   in   f  o  m  ,  ,  Cn  a  f  e  '  s   iiscussion  p'-o- 
vides  3   partial   explanation   for   Ihs   variability  W9 
foand.     Cnafe   notes   tnat   "centars  of  interest,  as 
expressed   in    extended   sentencss,   show  significantly 
greater   variation   in   the   anount  of  information  tney 
contain   tnan   do  focuses  of  consciousness,    as  expressed 
in   idea  units       (Cnafe,    1952,    p.    33).     Cnafe   uses  the 
term  "centers  of  interest   "   to   refer   to  linguistic 
units  wnich  consist  of  a   seriesof ^focuses  of  cons- 
ciousness and   allow  js   to   express   information  that 
would   be   too   extensive   for   expression   in   a   single  idea 
unit.      Cnafe   also   notes   that   "a   particular   span  of  * 
experience  does  not  necessarily  dictate  a  particular 
division   into   centers  of  interest*"   Thus    the  same  indi- 
vidual  may  divide   the  "same"   informa-tion   in  different 
ways   au  different   times,   either  by  incorporating  dif- 
ferent  kinds  of  information   into   a   single  center  of 
interest,  or    by  dividing   larger  chunks  of  ipfcprmation 
into  different  sequences  of  such  units. 

Some  of   the   variability  we   found   in   the   ways  the 
"same"    information   was  expressed  differently   in  the 
oral   transcripts   anj   the   written   compositions  may  thus 
be  explainej   by  the  notion   that  speakers  and  writers 
make  judgments  in   the   process  of  verbalization,  and 
tnat  such  judginents  can   vary   from  speaker   to  speaker, 
or   from  writer   to   writer   at^  different   times.  Given 
th3t  the   two    instances  of  verbalization   in   our  data 
w?^*e   also   addressed    to   different  audiences,   occurred  in' 
very  different  contexts    (an   oral  ^int^rview  and   a  writ- 
ing  assessment),   and   were  conveyed   in    two  different 
mejiuns,   it    is  not   surprising   thsft   a   great  deal  of 
variaoility   existed.     This  variaDility  prevented  a 
strict  one--to-one   accounting   of  the   "idea   units"  occur- 
ring  across    the  oral   and   written   sets  of  data. 
Instead,   we    independently  identified   lari'.er  sequences^ 
of  units,   reconciled  discrepancies,    and   adopted  a 
descriptive   analysis.     For  most  of  the  students,  how- 
ever,   it  was   clear   that  the   ideas  that  occurred   to  then 
during   their   initial   reading   and   interpretation  of  the 
prompt   were   used   in   one   form  or   another   in   the  essays. 

In  discussing   the   resultiJ  of  ou'^   analysis,    it  is 
imoortant  to   point  out   that   the  analysis   was   not  based 
on   a   single   reading   of  tne   pro, not.      Several    tines  dur- 
ing  the   interviews,   students  were  prompted    to   look  back 
at   tne   prompt,   read    it   again,    and   to   tnink  if  there 
were  any  a-sditional   things   tnat  came   to  mind.  Even 
tnougn  the  students  had   been   selected   by   their  teacners 
as  students   wno  would   be   likely  to   be  comfortable   in  an 
interview  situation,   several   of  tne   students  required 
repeated   prompts  before   they  could   focus   on   a   topic  for 
their   essay.      It   is   entirely  likely   that  tnis  pronpting 
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hai   an    effect  on   tne   students'  process   as   tney  callsi 
upon   personal   experience   and   knowledge  of  oonjitlons  at 
tne  school  'to   T^eet   tne   perceived   de-nands  of  the  task. 
Tnjs   the   '*  ideas'*    that  were  generated    in   the  oral  inter- 
view night   not  have  been   precisely  the   sane   ideas  the 
the   students   w o u  1  J   have   generated   had    they  been  readin-j 
the   p^o-npt   in   i  s--  1  ation   before   writing   tneir  inter- 
views. 

I::   is   also    i-nportant  to   point  out   that  several  of 
the   students   comnented    that  tney  felt   they  nad  an 
advantage  over    tne   other   students  in    the  assessnent, 
because   they   had   ti^ne   to   think   about   their  composition 
in  advance.      It   is  difficult   to   calculate   the  effects 
of  interviewer   prompts   and   increased    planning   tine  on 
their    perfornance   in   the   assess nent. 

Clearly,    the   limitations   in  any   sort  of  protocol 
analysis  include   the  possibility  that  researchers 
interfere  with   the  very   process   they  are  attenpting  to 
study.      Increased   training   in    the  "think-aloud"  pro-- 
cedure,    so    that  pronpting  would    be  less  necessary, 
would    be  advisable   in   any   future  study.      In  addition^ 
interviewer   pronpts   snould   be   standardized,   so  that 
variation   in   individual    interviewing   style   is  ninln- 
i  ze  d  . 

Although   the   conclusions   that  can   be   drawn  fron 
tnis   analysis   are   sonewhat  United,    the  data  provides 
3Dne   insignt    into    the  ways  neaning   is   constructed  as 
students   fit   tneir  own   knowledge  and    experience   to  per- 
ceived   tasK   denands,   diverging   at  tines   fron  the 
intended   neaning   of  the   authors  of  a   p-onpt   in   the  pro- 
cess,   and    transforning  neaning   even    further   as  they 
inplenent   their   ideas   in   conposing  written   text.  For 
tnese   students,   however,   the   task  of   responding   to  the 
text  of  a   pronpt  and   conposing   a  written  co.nposition 
were   closely   related".     The  data   indicate   that  both 
reading   and   writing  are   intertwined   in   the  conposing 
process,    that  sone   variation   in   individual    task  con- 
structions can   be   traced   to  ways   students  interpreted 
pronpt   segnents,   and   that   sone   variation   is   the  result 
of  rhetorical   c ho  ices   the  student  nakes  when  conposing 
a  response.      Tne  data   also  indicate   that   the  students 
in   the   subsanple  nade  various    interpretations  of  pronpt 
seg-nents,    and    that   these   interpretations   were  related 
to   the   rnetorical    strategies   they  adopted    in  coTiposing 
a   response    to    tne  prompt. 

cow:lu3i 

In   Part   II  of  tnis   report   we  discussed   the  process 
of  pronpt   construction   in   a  pronpt   developnent  session 
involving   tnree   teachers.     Tne   original   text  of  the 
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p-onpt;  ^as  introduced  by  one  of  thi  teachers  in  the 
f  ol  losing  for-n: 


Tnere   are  currently   plans  ^einj   forTned  for 
i.Tiprovenents   to   Central   High   School   as  a 
result   of  a   State   of  California   grant  for 
School    Inprove-nen  t  .     Tnere   are  many  co-i'nit- 
tees  working  on   plans   for    school   in  pr  o  v  e  r.en  t  . 
Identify  three   im  pr  o  v  e-n  en  ts   you  would   liKe  to 
see  Tiade  at   Central,   telling   why  you  feel 
each  1:^1  p'- oven  en  t  needs   to   be  nade  and  waat 
you  would   like  done. 


Tnis  version   was   i  ni   ed  i  a  t  e  1  y  changed   because  it 
was    found    to   be   too   structured.      The   teacher   cotj-ti  en  t  e  d  : 


.    .    I  didn't  wanna   go   into   the  whole   •    •  grant 
.    .   tnat  we  got   .    .    I  think  .-nose  of  the 
Kids  know  about   it    .    .    •   now   .    .  IWe 
structured   it  too  much   •    .    so    I'm  gonna 
change   it  as   I  read    ....  ^ 


Tne   original   p'-oript   author    in   a   subsequent  conTiunica- 
tion   to   researchers  offered    further   explanations  for 
why  tne  original   wording  of  the   p^'o-npt   was  changed: 


It  struck  Tie,  as  I  beg 
unnecessarily  formal  a 
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Th  e  authors 

of 

tne  promp 

t  did 

not  feel   that  tne   "specific*'   context  they  originally 
had    in  mind   would    present  a   problem   to   students  who 
weren't  privy  to   recent  discussions  within  committees 
re'iewing   proposals   for   use   of  recently  acquired  grant 
funds.     They  appeared   satisfied   that   they  had  suffi- 
ciently generalized    their   initial   context   to   allow  for 
a  wide  range  of  possible  responses   from   the  students. 

It   is   important   to   emphasize  at  this   point  tnat 
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tne   3JtnD'-3  of  tne   p-oapt  began  with  tne  description  of 
a   V2ry   specific  context,   3nd   ware   attetiptin;   to  no^e 
awiy   froi   specification  of  t^at  context   toward  a 
si-npler   subject.      Tne   result  of  tneir   efforts  was  san- 
tence    1,   containing   a   residual   core  of  infor. nation 
re-noved    froT)   tne  specific   facts   and   circunstances  sjr- 
roundinj^  it. 


Many  different  suggestions  for  improve-nent  of 
Central   High  School   have  recently  been  -nade. 


Teacners  con-aonly  construct  pro-npts   to  include 
initial   "  context  ualizing'^  or   backgrounding  material  • 
In   T)dny  cases,   as   was   the  case   here,   teachers  vie^ 
these  con tex tu al izing   statements  as  secondary   to  the 
tasK   indicated   in   the   rcnainder  of  the   pro-npt.      It  is 
not  surprising   therefore,   that  the  additional   group  of 
holistic  scorers,   like   the  developers  of  the  pro-npt  , 
did   not  p'-edtct  the  consequences  of  SK      It  would  be 
erroneous    to   assu-ne,    however,    that  all    students  would 
view  tnese   contextualizing   state  n  en  ts   as  s^rcondary 
given   tne  conditions  of  an   assessment  situation.  Sen- 
tence   1   is   recognizable  as   a  condensed   version  of 
snared    information.     vTnus,    it   is  understandable  wny 
sone  students  woulj   construe   31   as   their  cu2    to  recon-- 
stru?.  t   the   referenced   context  and   proce-?J   to    w-ite  fro-n 
tne  perspectives   existing  within   that  context.  Stu- 
dents wno  were  p-eoccupied   witn   the  reconstruction  of 
tne   "given*'   context   for  Si   found    it  difficult  to  adopt 
a   personal   perspective  and   tended   to  define  their 
"voice"   for   the  purpose  of  writing   tnis  essay  in  tc^rms 
of  opinions  snared   by  others. 

Tne   analysis  described   in   Part   II  of  this  report 
indicates   that  the   target  sample  of  students  made  dif- 
ferent  interpretations  of  segments  of  the  prompt. 
These  differences  have  the  potential    for  affecting  the 
evaluation  of  student  performance  If  they  lead   the  stu- 
dents  to  adopt  strategies  for  writing  which  do  not 
match  the  expectations  of  the  evaluators.     In   the  case 
of  the   students  In  this  sample,   ideas  generated   in  the 
course  of  reading   the  prompt  appeared   in  some   form  in 
the   writteri   essays.  ^  In   addition,   the   strategies  for 
writing   adopted   by  several  of  the   students  in   the  sub- 
sample   involved    in   large  measure   the  sort  of  "evalua- 
tion"  strategies  described  above.     In   this  data, 
"evaluation"   did   not  have  the  positive  effect  that  it 
is  described   as  having   in  narrative.      If  anything,    tr.  e 
reverse   was   true,   for   the  students  often   failed  to 
present  an  "objective"   case,   and   relied   instead  on  tne 
rnetorical  device  of  attributing  opinion  to  a  second 
person   ("my  friend   says")   or   to  a  generalized  group 
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("xi:3s   say").      One   could   argj*,   that  these  strategies 
depanjed   in   part  on   the   pro-npt,   sinae  "de.'^cribinj  a 
prot)le-n"    is   a   potentially  controversial    suiDject,  at 
least   fro-n  the   student's   point  of  view.      The  pronpt 
requires  students  to   ^'criticize'*   their   school,  which 
may  lead   sone  students  to   select   "safe"*su&jects  and  in 
general   fail   to   adopt   a   strong   personal  voice. 

Tne   extensive   evaluative  co.mientary   in   the  student 
essays   reflects   tneir   iack  of  confidence   in  the 
relevance  and    appropriateness  of   their  own  opinions. 
In  our  analysis  of  the  pronpt»   we   eTiphasized   that  Si 
contained   the  basis  for  misconceptions  on   the   part  of 
students  of  wnat  tne   task  required.   We   are  describing 
these  interpretations   as  misconceived   only  insofar  as 
they  did   not  meet  the  expectations  of  the  develoi^rs 
for   a   response   to   the   prompt.     The   students*    intr  pre- 
tentions of  the   prompr.   should   not  be  considered  misin- 
terpretations.   Rather,   they  should   be  viewed   as  varia- 
tions of  possible  interpretations   that  are  inherent  in 
the  proTipt.     Many  of   the  students   appeared   to   feel  that 
their  discussions  should   reflect  previous  discussions, 
and   several   of  the  students   felt  it  was   necessary  to 
justify  their   choice  and/or   give  reasons  wny  they  felt 
their  choice  was  ^generally"   relevant.     Tne  following 
examples  illustrate  how  one  student,   in   an  effort  to 
account   for   all   possible   interpretations  of  the  phrase 
"giving   reasons  for   your   choice,"    tried    to  acconplis'n 
two  of  the  evaluative   tasks   implied   in   the   phrasa  "giv- 
ing  reasons  for   your  choice." 
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-H  1  *^!l££5.*   t^nis  problem  is  because 
wh  :?n   I  first  started  high  school  my  grades 
dropped   really  fast.      (Jay,  Essay) 


In    tne   first  example,    the   pnrase  "the   reason  I 
think"   is   followed  by  a   conditional  or   logical  causal 
relation  sis^nalcd  by   "when,**   "there  is,*   wnich   is  gen- 
eralized  by  the  use  of  the  generic   pronoun  "you^**  In 
the  second   example,   the  phrase  "the   reason   I  chose"  is 
followed   by  a   recounting  of  personal   experience.  Ttie 
former   accounts  for   a  generalized    relevance  of  the 
problem   Chosen,   and   the  latter   expresses   the  personal 
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Oar   study  suggests   that  context   plays   an  important 
role   in   both   the   reading   and    the  writing  process. 
Clearly,    a   context   for    the   first   sentence   (or   the  lack 
of   it)    influenced   the   students*  interpretations  of  the 
proTjpt   and    ultiiiately   the  way   they   for-nulated  their 
responsesin    the  written   essay.      However,    the   issue  is 
not  simply  one  of   "knowledge"   or    the   lack  of  it. 
Ratner,    the   issue   is   whether  or   not   a   context   should  be 
provided,    and   when.      If   a   proupt   is  intended    to  be 
open,    we   should   avoid   obscured   contexts   wnich  Tiay  lead 
soTie  readers   (sucn   as  Tiany  of   the   students   in  this 
study)    to   atteTjpt   a   reconstruction  of  the  authors' 
intended   frane  of  reference.      If   a   context  of  prior 
knowledge   is   considered    to   be   inportant,    as   of  course 
it   is    in   Tiany   assessnent  situations,    then  proTipts 
sn ould    clearly   specify   what   that  context  is. 
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Wnen   students   have   only  one   chance   to   write,  and 
are  judged   on   this   single  performance,    it   is  especially 
luportant  that   students   have  optiiial   conditions  for 
their    performance,    and    that   there  be   no  mismatcnes  of 
either    topic   or   task.      The   findings  reported    in   Part  I 
of  this  report  indicate   that   students   and  teachers 
disagreed  on  many  elements  of  task  interpretation.  In 
addition,  unstated   expectations   were   found    to  be  corre- 
lated  with   score.      The   analysis   reported   in   Part  II 
indicates   that  various   interpretations   were  made  of 
prompt   segments,   and   that  these   interpretations  were 
related    to   rhetorical   strategies   adopted   by   tne  stu- 
dents  in   composing   a  response.      Tne   findings  have  seri- 
ous  implications   for   current   p'-actices   in  assessment, 
since  mismatches  of  this   kind   can   result   in  distortions 
of   the   ways    individual    students   are   judged.      The  first 
step  for   both   the   student  and    the  evaluator   in  a  writ- 
ing  assessment   is   the   reading   and   interpratation  of  a 
pro'ipt.      Tnus,    the   features  of  a   pronpt   wnich  influence 
interpretation   are  of  critical   importance   for   both  the 
reliability   and    the  validity  of  our   assessment  meas- 
ures. 
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Teacher^ 

Period 


You  recently  had  «  short  writing  Assignment  on  the  following  topic; 

Many  different  suggestions  for  improvement  of    Central    High  School  have 
recently  beer.  Tuade.    Describe  one  problem  or  lituation  «t    Central  which 
you  feel  needs  correction  or  improvement,  giving  reasons  for  your  choice 
end  suggesting  one  or  mere  solutions. 

This  is  a  questionnaire  zo  find  out  how  you  feel  about  vritirg  in  general  and,  apecif i.cally , 
what  vD'ui  think  abcut  the  tepic  you  had  to  write  on. 

Please  circle  the  n-jsber  of  the  choice  which  best  reflects  ycur  opinion  about  the  state- 
ments in  tbe  questionnaire.     Each  question  has  tvz  extremes,  vhi^h  have  been  placed  on 
either  Didn  cf  a  scale.    Fcr  Ertir.tle; 

St  r  on  g 1 y  S  t  r ong 1 v 

Agref  Agree        Neutral        Agree  Agret: 


I  like  valtciHK; 


I  dislike  walKiiig 
to  scbrc' . 


If  vou  strrngly  agree  with  the  statement  or  the  left,  then  you  would  circle  nurber  I, 
If  v':u  stjrcn^ly,.  agree  with  the  state'^ent  on  the  right,  then  you  would  circle  nurher  5, 
If  ycu  dc  not  agree  with  one  of  the  statements  more  than  with  the  ether,  then  you 
wruld  cir:  1^  the  nj-ber  frr  the  neutral  cateRory  -  nurrber  3.    £c^^H£, JiLI^ ^ A. 
ir.evr^       V— ^  Ri'iec  cf  fh^^  acale  before  you  circle  your  choice. 

Please  n'.^  the  teacher  r^v;  if  y-^u  dc  not  urdGrstnnd  what  you  are  i''-  dc. 
Be  s-.m  yrur  n^ne  in,  At  the  t^■^p  of  this  pago. 
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Page  1 


Strongly 
Agree 


Agree  Neutral 


1.  1  know  of  •any  (•ore  than  3) 
problei««  which  affect  the 
•otire  achool  at  Central. 

2.  I  often  think  about  Impro- 
ving the  school* 

3.  When  I  think  of  problems 
lu  the  achool,  I  think  of 
thinga  which  affect  many 
or  all  of  the  students. 

4.  In  general,  Central  seeon 
to  have  few  problems. 

5.  I  like  to  suggest  ways  to 
help  the  school  become  a 
better  place  to  be. 

6.  I  think  this  topic 

■1 lowed  mR  to  voice  an 
opinion  about  an 
important  issue* 


7.      I  ani  glad  I  had  the  chanre 
to  tell  someone  about  the 
autjec  t  I  wrote  on. 


Strongly 
Agree  Agree 


I  know  of  no  problems  which 
affect  the  entire  school 
at  Central. 

I  never  think  about  improving 
the  school. 

When  I  think  of  problems  in  the 
school,  I  think  of  thinga  which 
affect  only  myself. 


In  general.    Central    seems  to 

have  many  problems. 

l^do  not  like  to  suggest  waya  to 
help  the  achool  become  a  better 
place  to  be. 

I  do  not  think  thia  topic  allowed 
roe  to  voice  an  opinion  about  an 
important  issue. 


I  am  not  Rlad  I  had  the  chance 
to  tell  someone  about  the  subject 
I  wrote  on. 
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Page  2. 


8.    The  topic  was  a  satisfying 


one  to  write  on. 


Strongly 
Agree 


<- 


Agree 


Neutral 


Agree 


Strongly 
Agree 


The  topic  was  not  a  satisfying 
one  to  write  on. 


9.    The  topic  was  more  pleasing 
to  write  about  than  nost 
othera  given  in  achool. 


<- 


The  topic  was  less  pleasing  to 
write  about  than  most  others 
given  in  school. 


10.  The  topic  was  easy  to 
read  and  understand. 


The  topic  was  difficult  to  read 
and  understand. 


11.  The  topic  was  easy  to 
write  about. 


The  topic  was  difficult  to  write 
about . 


12.  I  had  a  hard  time  choosing 
something  to  write  about. 


I  did  not  have  a  hard  time  choosing 
something  to  write  about. 


13.  I  think  I  wrote  better 
for  this  assignment  than 
I  usually  do. 


< 


4  b.' 


> 


I  think  I  wrote  more  poorly  for 
this  assignment  than  I  usually  do. 


Page  3. 


Strongly 
Agree 


14.     I  think  I        «  good  writer.  i 


15.  I  think  th«  topic  will  give 
a  good  Indication  of  my 
writing  ability. 

16.  I  enjoy  writing. 

17.  I  think  learning  to 
write  well  la  "n  important 
thing  to  learn  In  achool. 


18.     I  think  that  being  able  to 

write  well  will  be  Important  i 

to  Me  after  I  leave  high  ^  

school . 


Agree         Neutral  Agree 


Strongly 
Agree 


I  think  I  am  a  poor  writer. 

I  think  the  topic  will  not  give  a 
good  indication  of  My  writing 
ability. 

I  do  not  enjoy  writing. 

I  think  learning  to  write  well 
is  not  an  important  thing  to 
learn  in  school. 

I  think  that  being  able  to  write 
well  will  not^  be  important  to  me 
after  I  leave  high  school. 
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The  following  Itens  refer  specifically  to  the  topic  you  wrote  on  yesterday.  You  nay  go  back  to  the  first  page  and 
look  at  the  topic: 


Strongly 
Agree 


Agree 


Neutral 


Agree 


Strongly 
Agree 


1.     I  wrote  on  only  one  problem 

or  aituarion  in  responding  1 
to  the  topic.  ^ 


> 


I  wrote  on  more  than  one  problea 
or  situation  in  responding  to 
the  topic. 


When  Writing  on  This  Topic: 


Strongly 
Agree 


Agree 


Neutral  Agree 


Strongly 
Agree 


2.  It  ia  important  to 

describe  only  one  problea. 

3.  It  ia  important  to 
explain  why  the  subject 
chosen  is  a  problem 

at  Central. 


> 


It  is  not  important  to  describe 
only  one  problem. 

It  ia  not  important  to  explain  why 
the  subject  chosen  is  a  problem 
St  Central. 


^*     It  ia  important  to 

suggest  a  solution  to 
the  problem  or  situation. 

5.     It  is  important  to 

8uj,;e8t  more  than  one 
solution  to  the  problem 
or  situation. 


1  


It  ia  not  important  to  auggest 

a  solution  to  the  problem  or 
situation. 

It  is  not  important  to  suggest  more 
than  one  solution  to  the  problem 
or  situation. 
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When  Writing  on  Thia  Topic; 


Strongly 
Agree 


Agree 


Neutral 


Agree 


Strongly 
Agree 


6*     It  ia  iaportant  to 
point  out  that  Mny 
■uggestiona  for  inproving 
Central  High  have 
been  aade. 


It  is  not  important  to  point 
out  that  many  suggestions  for 
Improving  Central    High  have 
been  made. 


It  is  important  to 
relate  the  problem  or 
aituAtion  to  everyone 
in  the  school. 


It  is  not  important  to  relate 
the  problem  or  situation  to 
everyone  in  the  achool. 


It  ia  important  to  show 
how  the  problem  affecta 
the  person  doing 
the  writing. 


It  ia  not  important  to  show  hov  the 
problem  affecta  the  peraon  doing 
the  writing. 


9.     It  is  important  to  explain 
how  the  solution/correction 
of  the  subject  chosen  will 
improve     Central  High. 


<- 


It  ia  not  important  to  explain 
how  the  solution/correction 
of  the  subject  choaen  will 
improve  Central  High. 
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Page  1 


APPrNDiX  B 


Strongly 
Agree 


1.  The  topic  would  be  more 
pleasing  for  students  to 
write  about  than  moat  others 
given  In  school. 

2.  The  topic  %rould  be  easy  for 
students  to  read  and 
understand . 

3.  The  topic  would  be  easy  for 
students  to  write  about. 

I  think  the  topic  would 
give  a  good  Indication  of 
student's  writing  ability 


Agree 


Nc^utral  Agree 


Strongly  * 
Agree 


The  topic  would  be  less  pleasing 
for  students  to  write  about  than 
most  others  given  In  school. 

The  topic  would  be  difficult  for 
students  to  jead^  and  UAdersjtand. 


The  topic  would  b^  difficult  for 
students  to  write  about. 

I  think  the  topic  would  not  give 
good  Indication  of  student's 
writing  ability. 


I 


Page  2 


The^ foliowing  items  refer  specifically  to  the  tnplr.  You  may  go  bark  to  the  first  page  and  look  at  the  topic: 
H^lf  il       Mi's  yJL     i  ^J^^P  ^  * 


1.  It  Is  Important  to  describe 
only  one  problem. 

2.  It  is  Important  to  explain 

why  the  subject  chosen  is 
a  problem  at  Central. 


St  rongl V 
Agree 


Agree         Neutral  Agree 
2  5  A 


Strongly 

Agree 


 i 


/  


> 


It  Is  not  Important  to  describe 
only  one  problem. 

It  is  not  Important  to  explain  wK^ 
the  subject  chosen  Is  a  problem 
at    Central . 


3.     It  Is  Important  to 

suggest  a  solution  to 
the  problem  or  situation. 


< 


-  > 


It  is  not  Important  to  suggest  a 
solution  to  the  prohlen*  or 
sltual ion. 


^.     It   is  Important  to 

suggest  more  than  one 
solution  to  the  pr:jbl*^m 
or  situation. 


It  is  not     important  to  suggest 
more  than  one  solution  to  the 
problem  or  situation. 
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VII'^IL-KL^ ^-^ This  Topi r : 


It  is  important  to 
point  out  that  many 
suggestions  for  improving 
Central     High  have 
been  made. 


Strcingly 
Agree 


Agree 


Neutral  Agree 


St  rongly 
Agree 


\ 


It  is  not  important  to  point 
out  that  many  suggestions  for 
improving    Central     High  have 
been  made. 


It  is  important  to 
relate  the  pitciblem  or 
situation  to  everyone 
in  the  school. 

It  is  important  to  show 
how  the  problem  affects 
the  person  doing 
the  writing. 

It  is  important  to  explain 
how  the  solution/correction 
of  the  subject  chosen  will 
im£r  ()  ve   Cc  n  t  r  a  1      H  i  ^',h . 


 1  


It  is  not  important  to  relate 
the  problem  or  situation  to 
everyone  in  the  school. 


It  is  not  important  to  show  how 
the  problem  affects  the  person 
doing  the  writing. 


It  is  not  important  to  xplain 
how  the  situation/correction 
of  the  subject  chosen  will 
1 mp r  o v  e     Central  High. 
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Chapter  5:    Part  1 


UNEXPECTED  DIRECTIONS  OF  CHANGE  IN  STUDENT  WRITING  PERFORMANCE* 

Catharine  Lucas  Keech 

Introduction 

A  striking  problem  which  recurs  repeatedly  in  all  direct  measures  of 
writing  performance  is  the  instability  of  student  performance  from  one 
occasion  to  another.  Users  of  essay  tests  are  troubled  by  the  frequency 
with  which  individuals  and  even  whole  groups  may  fail  to  produce  consistent 
improvement  on  these  measures  over  time--in  some  cases,  even  after  concentrated 
writing  instruction.    This  phenomenon  has  proved  baffling  and  frustrating 
to  teachers,  researchers,  and  program  evaluators.    As  children  learn  to 
write  better,  their  scores  on  essay  tests  should  simply  go  up.    If  the  scores 
don't  improve,  we  may  be  forced  to  conclude  that  students  have  not  improved 
their  writing  abilities. 

Of  course,  one  must  examine  the  test  measures  used:  were  the  two  test 
occasions  truly  parallel  in  what  they  required  students  to  do,  and  were  the 
tests  scored  in  the  same  manner?    Testers  have  reported  improving  performance 
stability  substantially  by  improving  inter-rater  agreement,  phrasing  prompts 
carefully  to  make  demands  more  parallel,  taking  longer  samples  to  improve 
discrimination  of  the  measure,  and  taking  more  than  one  sample  at  each  test 
sitting.    (Diederich,  1974;  Godschalk,  et  al.,  1976;  Breland  and  Gaynor,  1979.) 
Some  unexpected  drops  in  score  can  be  explained  ,  in  addition,  by  those  uncontrolled 
conditions  traditionally  cited:    lack  of  motivation,  having  a  "bad  day,"  failure 
to  find  good  material  to  write  about,  or  external  distractions  as  when  a  hated 
subsitute  teacher  administers  the  posttest  on  a  day  of  campus  riots  in  the 
middle  of  spring  fever,  in  which  case  scores  may  drop  for  a  whole  class. 

All  such  factors,  however,  are  as  likely  to  affect  pretest  as  posttest 

scores:  although  program  evaluators  may  examine  test  conditions  more  assiduously 

*This  paper  presBnted  at  AERA,  New  York,  1982,  provides  the  theoretical  background 
for  the  research  reported  in  Part  II  of  this  chapter. 
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for  differences  which  might  account  for  disappointing  gains  in  posttest  scores, 
it  is  equally  possible  that  any  impressive  signs  of  improvement  from  pre  to 
posttest  may  be  due  to  factors  which  artificially  depressed  pre  test  performance, 
creating  an  inflated  picture  of  growth.    In  spite  of  these  uncertainties,  program 
evaluators  in  California    and  other  states  have  not  been  driven  to  abandon 
holistic  scoring  of  writing  samples  in  favor  of  the  more  reliable  but  less  valid 
multiple-choice  test  format,    (Spandel  and  Stiggins, 1980c)    But  adhe>^ing  to  the 
practice  of  using  actual  writing  samples  in  evaluating  student  growth  in  composing 
skills  requires  not  only  that  evaluators  control  test  conditions  more  rigorously 
but  also  that  researchers  begin  to  look  more  closely  at  what  changes  when  students 
write  on  different  occasions.    The  phenomenon  of  unexpected  cirops  in  scores  or 
failure  to  improve  in  statistically  significant  increments  on  direct  measures 
cf  writing  is  too  widespread  among  both  individuals  and  groups  to  be  adequately 
explained  either  by  random  factors  or  by  poor  instruction--especially  since, 
over  the  long  run,  improvement  does  finally  seem  to  occur,  whatever  the  ups  and 
downs  along  the  way. 

Two  aspects  of  the  development  of  writing  ability  have  so  far,  I  believe, 
been  inadequately  taken  into  account  by  researchers  and  evaluators  confronting, 
drops  in  performance  scores  on  the  part  of  students  who  should  perform  better: 
(1)  the  general  phenomenon  of  the  U-shaped  learning  curve,  or  the  existence  in 
all  complex  learning  of  plateaus  or  even  apparent  regression  as  learners  move 
from  one  level  of  competency  to  the  next  (Piaget,  1977;  Goodman,  1979;  Miller,  1980; 
Bever,  1982)- and  (2)  the  special  open-ended  or  ill-defined  nature  of  writing 
tasks  which  allow  (even  require)  the  writer  to  set  for  himself  to^some  extent 
the  difficulty  level  of  what  he  attempts  to  do.  (Bereiter  and  Scardamalia,  1978.) 

4/4 '-''-^ 


In  this  paper  I  would  like  briefly  to  state  the  case  against  current 
attempts  to  measure  growth  using  methods  developed  to  measure  absolute 
proficiency  levels— measures  which  may  be  misleadingly  affected  by  developmental 
unevennesses  in  perforr.ance.    Then  I  will  examine  in  some  detail  the  problem 
of  task  definition  for  all  longitudinal  research  in  writing,  explaining  how 
a  refined  discourse  typology  might  help  us  identify  and  describe  more  precisely 
what  students  do  differently  from  one  writing  occasion  to  the. next.    Finally  , 
I  explore  briefly  new  ways  of  asking  why  and  how  students  change  the  diffi culty 
level  of  writing  tasks  intended  by  the  tester  to  be  parallel  in  demands. 

Developmental  Unevenness  and  Current  Measures 

It  would  be  odd  if  learning  to  write  were  somehow  exempt  from  the  roller- 
coaster  character  of  learning  documented  by  cognitive  psychologists  (cited  above). 
In  fact,  many  writing  teachers  have  recognized  some  manifestations  of  the 
non-linear  nature  of  improvement  "in  developing  writers:  for  instance  at  the 
sentence  level,  the  appearance  of  fragments  that  so  often  accompanies  the  onset 
of  subordinate  clauses  or  noun  appositives;  or  the  run-together  sentences  that 
appear  when  students  start  developing  complex  ideas  using  sentences  that  stand 
in  close  opposition  or  apposition  to  one  another,  which  they  want  to  join  with 
an  introductory  adverb  rather  than  a  subordi nate  conjuncti on.    Similarly,  elabor- 
ately expanded  subjects,  or  the  use  of  abstract  nouns -as  subjects ,  frequently 
create  predication  errors  or  verb-jubject  agreement  errors  the  student  would 
have  no  difficulty  avoiding  in  his  earlier,  simpler,  syntax.    Analytic  error 
counts  and  even  holistic  responses  may  determine  that  these  writers-in-transition 
are  less  able  performers  than  before,  whert  actually  eaQh  of  these  flaws  can 
be  seen  in  context  as  signalling  movement  toward  a. higher  level  of  competency. 

Developmental  evaluation  is  further  complicated  when  the  complexity  of 
the  cognitive  task  being  attempted  causes  a  writer  to  shift  attention  away  from 


rhetorical  niceties  she  may  have  mastered  but  which  may  never  become  fully 

automatic,  and  so  breakdown  when  not  attended  to.    Freedman  and  Pringle  (1980) 

point  out  in  a  study  of  college  writers  that  conventional  rhetorical  evaluations 

of  writing  are  based  on  ractors  which  do  not  improve,  and  may  actually  suffer, 

when  a  writer  is  attempting  to  solve  a  communication  problem  that  lies  at 

the  outer  limits  of  her  cognitive  processing  capacity.    They  conclude: 

"In  evaluating  their  students*  writing,  teachers  seem  to  (lack)  a 
sens^e  of  the  complex  nature  of.. .development,  in  which  growth  in  one 
dimension  may  entail  momentary  awkwardness  in  another. . .When  teachers 
ignore  the  congitive  aspects  of  the  (writing)  process  and  focus  only 
on  the  rhetorical  features  of  the  completed  product, ...they  are  not 
in  a  position  to  anticipate  the  kind  of  breakdown  that  occurs  when 
the  intellectual  task  is  made  more  complex.    Further,. ..they  may 
actual  I'y  impede  intellectual  growth. (The)  short-run  effects  of  such 
evaluation  are  probably  to  encourage  students  to  operate  on  the  safe 
levels  they  have  already  mastered..." 

(Po  323) 

Tius,  students  may  write  more  awkwardly,  less -correctly,  less  fluently, 

either  because  they  are  acquiring  new  forms  at  the  word  or  sentence  level 

or  because  they  are  struggling  to  express  cognitive  functions  for  which  their 

current  language  forms  are  as  yet  inadequate.    A  third  developmental  struggle 

has  received  even  less  investigation  than  these  two:    how  the  student  defines 

the  rhetorical  task  at  the  text  or  discourse  level.    In  whole-text  planning, 

intend  to 

what  does  the  student  set  out  to  do,  and  how  does  he/do  it?   How  does  he 
conceive  of  the  artifact  he  is  about  to  create?    What  kind  of  text  does  he 
think  in  terms  of,  as  he  pursues  his  ideas  and  formulates  his  sentences?  To 
vhat  extent  does  he  conceive  of  the  whole  text  at  all? 

Recent  research  in  discourse  grammars  and  text  design  (Stein  and  Glenn,  19  79; 
Meyer,  1975,  and  others)  and  modern  discourse  theorists  h'ncludinq  Bain,  1890; 
Kinneavy,  1979  ;  Bri tton,  1975;  and  others:  see  D'Angelo,  1976)  have  reestablished 

the         importance    of  distinctions  among  genre  or  types  of  writing  which 
the  learning  writer  must  begin  to  differentiate,  mastering  a  variety  of  composing 
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strategies  to  accomplish  different  rhetorical  purposes^  ^ych  as  reporting, 

explaining,  persuading.     Deciding  what  he  wants  to  do  (tell  a  story,  explore 
his  feelings,  change  someone's  course  of  action,  etc.)  and  selecting  the 
appropriate  composing  strategies  to  accomplish  these  aims  are  two  important 
parts  of  representing  or  constructing  the  writing  task--two  processes 
particularly  subject  to  the  writer's  stage  of  development,  which  may  interact 
1n  confusing  ways  with  the  writing  assignment.    It  is  in  the  areas  of  text 
design,  rhetorical  purpose  and  strategy,  that  the  ill-defined  nature  of  the 
writing  task  comes  into  play,  creating  a  particular  set  of  problems  for 
measuring  development  in  writing  ability. 
The  Ill-Defined  Writing  Task 

It  is  crucial  in  evaluation  research  to  distinguish  between  the  given 
task  or  actual  text  of  the  writing  assi gnment— wi th  its  particular  set  of 
constraints  and  options,  expressed  or  implied,  and  with  its  virtually  infinite 
set  of  possible  real izati ons--and  on  the  other  hand  the  constructed  task  , 
the  set  of  constraints  actually  honored  and  the  options  actually  chosen  by 
the  student,  as  seen  in  the  text  the  student  produces.    Test  makers 

may  believe  they  have  carefully  constrained  a  particular  task  to  elicit  a 
particular  kind  of  response,  but  as  Murphy,  Carroll  and  Kinzer  (1982)  have 
demonstrated,  students  are  capable  of  creating  totally  unexpected  task  versions 
or  notions  of  what  they  are  "supposed  to  do."  (See  also  Keech  and  McNelly,  1982.) 

Unlike  other  performance  areas  —for  instance,  music,  where  a  learner 
is  given  a  more  difficult  piece  to  play,  or  mathematics,  where  both  teacher 
and  student  are  aware  when  a  more  difficult  problem  is  offered,  or  sports 

competitions  where  nore  advanced  dives  earn  more  points—  composition 

has  no  such  well-defined,  easily  ranked  tasks  or  gradations  of  performance 

levels  that  can  be  controlled  by  the  teacher  or  evaluator.    It  is  possible 
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to  give  students  the  same    assignment  on  two  different  occasions  and  have  them 

construct  such  different  composing  tasks  that  their  two  responses  are  hardly 

comparable.    The  student  might  appear  an  expert  on  the  first  writing,  while 

performing  like  a  novice  as  he  attempts  on  the  second  writing  a  far  more 

complex  problem.    Sufficiently  open-ended  assignments,  such  as  ''Write  about 

a  favorite  object,"  may  be  offered  again  and  again  over  all  the  years  of 

a  writer's  career,  from  first  laborious  printings  to  his  adult  years  of 

proficient  composing;  the  actual  task  difficulty  will  increase  1n  proportion 

to  his  writing  ability.    The  very  ability  to  define  a  task  more  richly 

and  complexly  may  be  one  of  the  most  important  writing  skills,  developing  slowly 

along  with  syntax  and  sense  of  oarag^aphing. 

Bereiter  and  Scardamalia  (1^79),  recognizing  that  writing  tasks  are 

inherently  ill-defined,  are  unconcprned  when  changes  in  young  writers  are 

not  reflected  in  conventional  scores  on  their  writing: 

"...(Our)  most  successful  experiments  so  far  in  affecting  children's 
composing  processes  have  not  led  to  discernible  overall  improvements, 
as  judged  by  impressionistic  ratings. . .This  would  be  discouraging  if  the 
purpose  were  to  get  children  to  do  a  better  job  of  pursuing  the  same 
goals  as  before.    When,  however,  the  purpose  is  to  get  children  to 
tackle  problems  they  have  not  tackled  before,  such  impressionistic 
results' are  immaterial.    It  is  what  they  are  doing  differently  that 
counts,  not  how  well  they  are  doVng  it  compared  to  how  well  they 
previously  did  something  of  a  different  sort."        (pp.  83-84,  ms.) 

Evaluators  trying  to  document  growth  or  development  in  writers  confront 
a  choice:  either  narrow  task  constraints  sufficiently  in  the  writing  assignment  to 
keep  writers  doing  the  same  kind  of  thing  on  trials  1  and  2,  or  begin  to  find 
out  how  to  describe    what  children  do  differently  from  one  occasion  to  the  next, 
how  to  interpret  that  difference,  and  finally  how  to  measure  the  development  it 
impl ies. 
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so  far.  evaluators  have  quite  properlv  concentrated  on  trying  to  mke 
task  demands  similar  for  repeated  writing  samples  used  to  measure  improvement 

over  time.  "^"^""^ 

that  changes  In  performance  abilities  may  he  obscured  if  a  student  is 
encouraged  to  write  a  personal  experience  reminiscence  on  one  occasion  and 
an  argumentative  essay  on  another:    If  nothing  else,  it  Is  difficult  to 
compare  these  responses  quantitatively,  scoring  them  under  the  same  rubric. 
At  least  the  very  rough  distinction  between  narration  and  exnositlon  is  typi- 
cally made,  with  many  program  evaluators  eschewing  the  earlier  practice, 
associated  with  the  Bay  Area  Writing  Project  model  of  holistic  scoring, 
of  allowing  students  to  respond  to  a  writing  stimuli  using  almost  any 
form  of  writing,  sometimes  including  poetry. 

Two  dangers  are  inherent  In  attempts  to  narrow  task  constraints  to 
reduce  variation  In  task  construal  for  purposes  of  Improving  comparison  of 
samples,  however.    The  first  is  simply  that  the  more  text  testers  add  to 
the  writing  assignment,  the  less^ guarantee  they  have  that  students  will 
read  and  correctly  interpret  all  of  the  guldelines-in  the  extreme  cases, 
students  may  either  ignore  a  lengthy  set  of  instructions,  or  may  become 
so  embroiled  In  working  out  exactly  what  the  tester  wants  that  they  are 
distracted  from  their  central  task  of  trying  to  generate  meaningful,  coherent 
text.    The  second  danger  appears  only  when  test  instructions  are  so  clear 
and  so  good  that  they  actually  succeed  in  narrowing  the  task  and  making 
student  responses  easy  to  compare  and  score  because  the  student  is  being 
asked  to  do  so  little.   Writing  tests  can  be  made  to  be  little  more  than 
direction  following,  requiring  students  to  show  proficiency  in  the  use  of 
certain  written  conventions,  but  falling  altogether  to  test  the  student's 
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ability  to  actually  "author"  a  text,  as  Moffett  (1979)  identifies  the  highest 
composing  skills.    Too  narrowly  specified  tasks  maximize  control  over 
what  is  attempted  but  may  invalidate  the  writing  test,  since  the  test 
no  longer  reveals  how  the  student  frames  the  rhetorical  problem,  sets 
goals,  chooses  strategies,  juggles  constraints,  revises  inner  speech— 
in  short,  really  composes. 

Ironically,  it  may  be  by  looking  at  samples  from  assessments  in 
which  students  were  given  maximum  freedom  to  choose  their  purposes  and 
strategies  in  response  to  a  topic  idea  that  we  are  best  able  to  discover 
what  kinds  of  writing,  what  discourse  options  are  most  likely  to  be  elicited 
by  certain  topics,  regardless  of  instructions  to  the  writer.    The  early 
BAWP  assessment  model,  as  represented  by  the  six-year  hodge-Dodge  of  topic 
types  used  at  Sir  Frances  Drake  High  School  (see   Table  1),  while  generating 
scores  that  soar  and  dip  in  unpredictable  ways  from  year  to  year,  provides 
a  rich  data  source  for  discovering  patterns  of  task  construal  in  response 
to  different  kinds  of  topics  as  well  as  in  relation  to  grade  level  or 
experience.  (See  Table  1,  p.  503.) 

The  Drake  Longitudinal  Sample 


The  Drake  teachers  who  composed  the  test  questions  shown  in   Table  1 
deliberately  made  them  as  open  as  possible    for  several  reasons „    They  we 
interested  in  how  students  might  construe  the  task:  they  wanted  to  know 
whether  their  instructional  program  had  succeeded  in  teaching  students 
the  large  range  of  strategies  that  might  be  used  effectively  to  solve  a 
particular  rhetorical  problem.    Since  they  were  attempting  to  teach  many 
forms  of  writing,  they  did  not  want  a  restriction  on  the  test  to  suggest 
that  only  one  of  these  kinds  was  valid  or  important. 
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Further,  they  were  unconcerned,  in  this  pioneering  effort,  to  make  tasks 
similar  from  year  to  yearo    Initially  their  goal  was  simply  to  create  a 
stimuli  that  would  allow  students  to  begin  writing  quickly,  with  maximum 
enthusiasm  and  ample  material  for  composing;  hence,  the  emphasis  on  personal 
feelings.    Later  they  wanted  to  see  how  well  students  wrote  evaluative 
or  argumentative  pieces  (1976  and  1977.)    In  the  last  year  (1978)  the  topic 
provides  a  complex  blend  of  possibilities,  allowing  students  to  focus  on 
either  an  experience,  a  person  or  thing,  or  an  idea--a  change  in  themselves^ 

The  teachers  were  well  aware  that  the  1973  topic,  "Write  about  an 
event  .  .  .  ,"  was  likely  to  evoke  imaginative  narratives,  while  the  1976 
topic,  "Name  one  invention  we  would  be  better  off  without..."  would  invite 
argument  or  exposition„    They  were  neither  surprised  nor  di spleased, however, 
when,  in  1973,  many  students  wrote  exposi tori ly ,  choosing  to  identify 
and  comment  on  an  event  they  wanted  to  witness,  rather  than  to  relate  it 
as  if  they  had  witnessed  it;  or  when,  in  1976,  one  student  demonstrated 
the  evils  of  television  with  an  expert  short  story,  wHle  another  wrote  a 
letter  to  Henry  Ford  in  heaven,  berating  him  for  having  contributed  to  the 
death  of  the  writer's  parents  in  a  car  crash. 

Three  longitudinal  samples  were  constructed  from  the  Drake  data  base, 
as  shown  in  Figure  1*,  each  containing  ninth  through  twelfth  grade  samples 
from  30  students.    When  the  four  paperS  of  one  student  .are  gathered  in  a 
case  study  folio,  it  is  ininediately  clear  that  students  have  in  fact  done 
different  things  on  different  rounds  of  assessment,  causing  their  holistic 
scores  to  change  in  surprising  ways.^  Not  only  does  the  change  in  topic  from 
year  to  year  invite  different  kinds  of  writing,  but  students  appear  to  set 
up  different  sorts  of  tasks  that  are  not  determined  by  topic  differences, 

*  Page  505. 
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but  are      allowed  by  the  writing  instructions.    It  is  precisely  the  lattitude 
allowed  students  that  makes  this  sample  of  papers  so  useful  in  giving  us 
an  idea  of  the  repertoire  of  text  designs  or  discourse  schemas  high  school 
writers  have  available  for  solving  a  number  of  different  writing  problems. 
In  one  sense,  the  Drake  sample  represents  the  worst  that  could  happen  in 
an  assessment  which  is  attempting  to  measure  improvement  in  writing  from 
one  occasion  to  the  nexto    By  representing  such  an  extreme,  the  sample 
allows  us  to  investigate  the  primary  problem  inherent  in  all  longitudinal 
writing  research  to  a  greater  or  lesser  extent:  our  inability  so  far  to 
account  for  task  complexity,  or  to  compare  fairly  the  difficulty  level  of 
what  a  student  attempts  on  different  occasions. 2 

A  system  of  task  descriptions  is  needed  to  determine  whether  or  not 
two  compositions  represent  the  same  or  different  types  of  tasks:  which 
purposes  does  the  student  appear  to  be  aiming  for?    Which  strategies  does 
he  use?    Only  after  researchers  are  able  to  describe  and  classify  the 
apparent  underlying  discourse  schemes  students  use  will  they  be  able  to 
distinguish  between  a  student's  mastery  of  old  tasks  and  his  novice  attempts 
to  accomplish  new  tasks.    A  refined  discourse  typology  appears  essential 
to  the  development  and  testing  of  hypotheses  about  the  relative  difficulty 
levels  of  what  is  attempted,  ultimately  allowing  predictions  about  which 
kinds  of  tasks  are  learned  first,  which  may  follow,  and  what  various 
SDontaneous  task  construals  may  indicate  about  a  given  student's  particular 
level  of  development. 

A  Discourse  Typology  for  Developing  Writers'  Texts 

Available  discourse  classifying  systems  proved  inadequate  for  describing 
the  student  texts  in  the  sample,  possibly  because  most  of  them  represent 
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what  might  be  called  ideal  text  types,  the  products  of  accomplished  adult 
writers  rather  than  of  students  who  may  be  only  approximating  these  discourse 
forms,        Further,  existing  typologies  offer     three  to  four  global  categories 
which  perforce  must  obscure  almost  as  many  differences  between  texts  as  they 
are  able  to  identify.    Just  as  Kinneavy  suggests  a  distinction  between  the 
aims  of  discourse  and  the  modes  of  discourse, ^  so  I  found  it  critical  to 
separate  discourse  function  from  rhetorical  strategy.    To  classify  texts 
according  to  function  is  to  ask:  How  does  the  text  function  for  the  reader? 
How  might  the  reader  characterize  the  writer's  underlying  purposes,  on  the 
basis  of  the  whole  effect  of  the  piece?    To  classify  strategy,  the  reader 
shifts  attention  from  the  whole  to  the  parts:  What  language  structures  does 
the  writer  use  to  accomplish  her  purposes?    The  final  classification  of  the 
text  then  is  in  terms  of  both  function  and  strategy:  what  was  done  and  how 
was  it  done? 


Figure  2*  shows  the  three  discourse  functions  found  in  the  sample, 
together  with  their  parallel  strategies.    Although  each  strategy  appears 
to"belong  to"a  particular  function,  the  strategies  are  separable  from  the 
functions,  not  merely  by  analysis  but  also  in  practice.    "What  to  do?*' 
and  "How  to  do  it?"  are  genuinely    discreet  questions  which  a  writer  may 
choose  to  consider  separately  during  composing.    On  the  other  hand,  some 
traditional  discourse  schemes  strongly  associate  a  particular  strategy 
with  a  given  function,  so  that  a  writer  In  choosing  to  create  a  text 
according  to  that  scheme  makes  his  function  and  strategy  choice  in  one 
stroke;  certainly.  If  he  wishes  to  tell  a  story,  he  seems  bound  to  use 

a  narrative  strategy.    Yet  he  might  also  separate  narrative  strategy 

instead 

from  Its  usual  function  and  use  it/to  reveal  an  entity  or  to  express  an 
idea. 
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The  separability  of  function  and  strategy  becomes  clearer  as  one 
contrasts  the  descriptions  of  the  functions,  which  are  organic,  pertaining 
to  the  whole  text,  with  the  descriptions  of  the  strategies,  which  are 
analytic,  pertaining  to  the  nature  and  arrangement  of  the  parts. 

To  identify  how  the  text  functions  as  a  whole,  the  reader  assesses 
his  own  response:  what  is  he  left  with?    For  discourse  function  Type  I, 
the  entire  text  appears  to  have  no  other  function  than  to  "tell  a  story," 
that  is,  to  dramatize  an  event  or  sequence  of  acts/events  that  occurred 
in  a  single  time  frame.    A  change  of  some  sort  occurred  over  tine,  and  this 
diange  is  the  focus  of  the  discourse.    In  discourse  function  Type  II,  the 
text  functions  to  reveal  the  nature  of  an  entity  (person,  object,  place,  etc.) 
which  retains  its  identity  and  essential  properties  over  time.    For  discourse 
function  Type  III,  the  text  reveals  or  expresses    an  idea  or  relationship, 
an  abstraction  from  concrete  events  and  entities  (beyond  the  linguistic 
abstractions  which  merely  serve  to  name  them),  a  logical,  analogical,  or 
tautological  construct  that  exists  not  in  time  but  in  mind.  The  same 
real  world  events  can  provide  the  given  material  for  texts  that  function 
entirely  differently.     The  role  of  Winston  Churchill  in  World  War  II 
could  be  dramatized  as  pure  story,  the  events  related  in  a  narrative 
sequence  as  entertaining  and  otherwise  unedifying  as  any  suspense  tale; 
or  the  story  could  be  told  as  a  portrait  of  the  man-rather,  a  personal 
history,  with  its  emphasis  on  the  hero's  character^  could  be  made  from  the 
same  events.    Finally,  many  sorts  of  analyses  of  the  events  in  which  Churchill 
figured  could  be  made  in  support  of  a  variety  of  assertions  about  cause 
and  effect,  the  nature  of  war,  the  potential  of  one  individual  for  historic 
impact,  etc..    creating  Function  III  discourse  about  ideas  or  the  meanings 
and  interpretations  of  events  or  evaluations  of  people  and  things. 
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To  some  extent  these  differences  in  function  appear  at  first  to  be 

simply  a  matter  of  choice  of  strategy:  narrative,  descriptive,  or  commentary. 

In  fact,  the  strategies  listed  here  are  so  widely  used  to  express  their 

parillel  functions  that  a  departure  from  convention  is  noteworthy,  in  some 

cases  creating  a  distinct  genre,  in  other  cases  a  failed  piece  of  writing. 
It  is  no  wonder  then  that  strategy  and  function  have  so  often  been  combined 

or  confused  by  theorists  who  sought  a  single  basis  on  which  to  classify  texts^ 

or  that  theorists  who  classified  according  to    function  assumed  that  strategy 

natural ly' followed.    This  confusion  may  have  resulted  in  part  not  only  from 

similarities  in  the  terms  used  to  describe  function  and  strategy,  but  from 

the  use  of  the  same  terms  to  describe  both  whole  strategies  and  individual 

propositi onSo 

An  adequate  definition  of  strategy*  or    choice  of  text  design,  must 
distinguish  between  core  propositions,  forming  the  spine  of  the  text,  and  other 
statements  which  elaborate,  extend,  provide  background  for  these  core 
propositions.    Linguists  have  made  this  distinction  for  narratives  or  stories 
(labov  and  Walletsky,  ]9       Hopper,  1977);  here  I  have  extended  it  to  other 

discourse  types.    I  have  often  been  disconcerted  by  efforts  to  describe  texts 

versus  versus 
as  narrative  /    uti^cripti vF7    commentary  when  I  could  find  in  almost  every 

text  a  lavish  sampling  of  all  these  kinds  of  writing.    A  second  glance  at 
most  texts  further  reveals  that  the  differences  between  them  lie  not 
merely  in  relative  numbers  of  one  kind  of  proposition  or  another.  Rather, 
the  key  to  text  design  strategies  seems    to  lie  in  the  relations  of 
these  different  kinds  of  elements  to  one  another. 

Figure  3*shows  how  a  strategy  can  be  identified  by  reference  to  which 
kind  of  propositions  form  the  core  of  the  text.      In  narrative  strategy, 

*  See  page  507. 
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for  instance,  the  core  propositions,  forming  the  spine  of  the  text,  are  a 

series  of  statements,  like,  "X  happened;"    *'X  is  happening;"  or  "X  happens;" 

the  latter  (simple  present  tense)  occuring  only  when  it  is  clearly  historical 

present,  not    habitual  aspect: 

Histori cal  Present:  Habitual  Aspect 

(reference  to  one  particular  (reference  to  a  general,  often 

occurrence)  repeated  occurrence) 

"The  footsteps  approach  my  door.  I  *'For  breakfast,  I  eajt  more 

fix  my  eyes  on  the  handle  of  the  door,  *  than  at  any  other  meal  of  the 

the  gun  steady  in  my  hand.  As  the  door  day.    I  drink  at  least  8  02. 

opens ,  I  shooto  A  body  crumples  to  its  orange  juice,  and  another  8  02. 

knees  then  topples  into  the  room.    With  milk,  down  6  pancakes,  4  eggs, 

slowly  dawn-'ng  horror,  I  reali2e  that  I  and  finish  up  with  toast  or 

have  shot  my  husband.. 0"  hash  browns,  with  cheese  and 

melon  in  season^" 

Although  there  may  be  flashbacks,  or  background  description  and  commentary, 
the  E  statements  at  the  core  mirror  the  order  of  events  as  they  occurred  in 
real  time,  forming  what  Hopper  (1980)  calls  the  "foreground"  of  the  narrative, 
with  D  and  C  statements  providing  "backgrounding. " 

E  statements  require  transitive  or  intransitive  verbs.  Statements  based 
on  predicates  which  include  l-inking  verbs,  the  verb  to  be,  or  adverbs  which 
indicate  habitual  aspect  ("X  usually  is...;"  "X  sometirr6s  does...;"    "X  used 
to  happen;"    "X  would  go  and  come..."  (modal  marks  aspect  in  past  tense))  are 
descriptive  or  D  statements.    These  propositions  appear  as  background  in 
narration,  but  can  be  used  to  form  the  spine  of  a    text,  in  which  case  the 
text  strategy  may  be  called  descriptive.    Note  that  these  statements  may  be 
arranged  to  reflect  a  chronological  sequence  of  events,  without  creating  a 
narrative  text  strategy:  "Every  day  I  get  up,  comb  my  hair,  brush  my  teeth, 

yell  at  my  little  brother    to  make  the  beds,  grab  a  bite  from  the  fridge,  \ 

\ 

and  race  off  to  the  soccer  field.    Once  there,  I..."    They  may  also  be  arra\ged^ 
to  reflect  spatial  orderings  in  reality.    Finally,  they  may  be  arranged 
associationally,  mirroring  the  writer's  thought  processes    and  bouncing  from 
one  idea  to  another  with  connections  visible  only  to  the  writer,  or  focally, 
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grouped  according  to  some  key  aspect  of  the  entity  being  described.  (See 
Freedman  and  Pringle,  1982.)    The  term  "entity"  is  used  loosely  here: 
such  abstractions  as  "my  summer  vacation"  may  be  treated  as  entities  having 
consistent  and  identifiable  characteristics,  and  may  be  written  about  using 
a  descriptive  strategy  by  presenting  "a  typical  day  in    the  course  of..." 
It  is  a  question  in  determining  function,  not  strategy,  to  wonder  just  how 
abstract  the  subject  of  the  piece  is:    when  does  summer  vacation  cease  to 
function  as  an  entityto  be  described  and  begin  to  function  as  an  idea  to 
be  explored?    Yet  it  is  strategy  that  most  often  provides  the  answer  to 
that  question,    thfe  arrangement  of  propositions  determining  that  a  writer 
has  crossed  tne  line  between  descriptic  and  commentary. 

In  Type  C  strategy,  common  to  exposition  and  argument,  C  propositions, 
or  commentary, form  the  core  of  the  text.    E  and  D  statements  may  appear  in 
large  numbers,  witn  several  E  statements  strung  together  to  form  a  mini-narra- 
tive, but  in  all  cases  descriptive  or  narrative  propositions  will  be  sub- 
ordinate to  the  C  statements  which  they  support,  demonstrate,  elaborate,  define, 
etco    In  addition,  there  may  be  C  statements  which  are  subordinate  to  the 
core  propositions,  that  chain  of  assertions  which  provides  the  main  thrust  of 
the  text.    The  core  propositions  can  be  arranged  associationally  or  focally, 
as  in  Strategy  B  (description),  or  they  can  be  arranged  hierarchically, 
according  to  their  logical  relations.    The  assertions  do  not  have  chronological 
or  spatial  connections  to  one  another,  though  they  may  be  arranged  to  comment 
on  chronological  progressions  and  so  appear  chronologically  ordered. 

C  statements  are  recognized  in  several  ways:  1)  the  verb  is  marked  by 
a  modal:  "X  should,  would,  could,  might,  will  happen...;"  2)  the  verb  is  marked 
by  a  negative:  "X  never  does...;"  "X  did  not  happen..;"  3)  the  main  proposition 
is  imbedded  as  a  noun  clause  of  indirect  quotation:"(Writer/X)  believes,  thinks, 
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feels,  wishes,  hopes  that..o;"  or  with  a  qualifying  expletive:  "It  seems 
that...;"  "It  is  clear  that..;"  3)  relationships  of  causality,  comparison, 
opposition  are  expressed  either  through  predication  within  e  clause: 
"X  is  like.*i"  "X  is  caused  by.,;"  "X  causes,, ;"or  by  coordinsting  or 
subordinating  conjunctions  and  Introductory  adverbs  relating  1ai*o  clauses: 
"If  X,  then  Y..;"    "Because  X,  then  Y..;"  "Either  X  or  Y..;"-"Not  X.  but  Y..;" 
"X  Is  not  true;  rather,  Y  happened;"   4)  the  predicate  defines  and  classifies 
the  subject  with  relation  to  other  things:  "X  4s  a  kind  of..;"  "X  1s  one  of 
two  kinds  of..;"  5)  the  verb  structure  Is  like  that  of  narration  or  description 
("X  Is/was;"  "X  does/did  ")  but  X  represents  an  abstract  subject:,  either 
a  non-count  noun  ("money;"  "water"),  a  hypothetical  construct  ("democracy;" 
"competence"),  a  feeling  state  or  emotion  ("love;"  "anxiety"),  a  nominal Iz^d 
verb  ("registration;"  "confusion;"  "obfuscatlon;"  "segregation"),  or  a  noun 
that  has  generic  rather  than  specific  reference,  as  shown. by  context: 

« eneric  referent:  specific  referpnt;  . 

e  family  In  America  today  Is  In  "The  family  down  the  block  Is  in 

danger  of  extinction."  danger  of  bankruptcy." 

"(The^oeople  who  believe  that  kind  of         "The  people  (who  ire)  cllnilng  Into 
propaganda  are  uneducated."  the  lifeboats  are  unafraid." 

Two  kinds  of  development  are  involved  as  a  writer  masters  these  text 
design  strategies:    the  acquisition  of  a  new  strategy.  Increasing  the  range 
of  choices  available  to  the  writer;  and  the  progression  from  novice  to 
expert  within  one  strategy.    Both  kinds  of  development  almost  certainly  Invo'lve 
transition  Stages  during  which  a  writer  may  produce  a  text  that  Is  difficult 
to  classify,  or  that  Is  successful  1n  neither  one  way  nor  anotherr  representing 
only  partial  mastery  of  a  text  grammar.    Figuring  out  where  a  student  Is  on 
the  roed  to  full  mastery  of  text  design  strategies  Is  further  complicated  by 
the  existence  of  sub-classes  of  ea.ch  of  the  three  strategies,  as  well  as  the 
possibility  of  deliberate  mixes  of  strategy  to  accomplish  certain  ends, 
d  "      488  CTn  - 
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Figure  4*represents  possible  Hnes  of  developront  and  acquisition  for 
the  three  strategies  described,  and  places  these  1n  the  larger  context  of 
other  discourse  strategies  not  classified  In  this  limited  typology  of  student 
text  types.    The  diagram  distinguishes  anrong  three  classes  of  strategies— 
pre-schema,  open-s-chema ,  and  closed-schcflw,  which  are  probably  acquired  in 
that  order  by  the  language  user  In  our  culture.    Pre-schema  strategies  are 

« 

most  comparable  to  Britton's  "expressive  writing/  relatively  idiosyncratic 
structuring  of  discourse  reflecting  the  writer/speaker's  own  flow  of  thought 
or  "inner'speech ."requiring  little  or  no  pre-planning,  allowing  the  sender 
merely  to  thi=nk  aloud  or  "compose  at  the  point  of  utterance."  (Britton,  1975.) 
I  coined  the  tenn  "pre-schema"  to  help  understand  papers  in  the  sample  that 
seemed  sti*ategy-less.    On  reflecting,  I  realized  that  following  one's  thought 
Is  Itself  a  composing  strategy,  but  one  which  does  not  recognize  and  use 
discourse  schemas  conventionalized  in  other  people's  discourse.  To  some 
extent,  of  course,  what  I  have  called  pre-schema  strategies  have 

become  conventionalized,  as  fiction  writers,  particularly^  present 
characters  who  narrate  in  their  own  voices.    This    kind  of  artful  artlessness 
In  the  hands  of  skilled  writers  mimics  the  natural  means  of  expression  of 
writers/speakers  who  are  not  able  or  do  not  choose  to  adopt  the  strategies 
developed  specifically  to  serve  the  needs  of  closed-schema  discourse.  In 
the  student  papers,  it  was  generally  easy  to  distinguish  between  spontaneous 
pre-schema  strategies,  usually  appearing  as  rudimentary  ruminations  on 
strongly  expository  topics,  and  simulated  pre-schema  strategies  in  which 
a  narrator  other  than  the  writer  introduces  him/herself  before  engaging  in 
a  self-exploratory  interior  monologue,  i  popular  solution  to  the  1974  and 
1975  topics  about  being  someone  or  something  other  than  yourself. 

*  See  page  508. 


The  open-schema  strategies  represented  in  Figure  4  were  posited  to 
account  for  another  group  of  student  papers  (as  well  as  literary  prototypes) 
not  classifiable  as  belonging  to  any  of  the  closed-schema  strategies.  The 
distinction  between  open  and  closed  discourse  schemas  is  made  by  Berelter 
&  Scardamalia  (1979)  and  others  as  an  alternative  to  distinctions  made 
between  oral  and  written  discourse.   Typically  oral  speech  occurs  in  open 
schemas.  where  one  does  not  structure  the  discourse  alone  but  is  aided  by 
conversational  partners.    Berelter  and  Scardamalia  point  out  that  one  of 
the  difficulties  for  children  learning  to  write  is  the  problem  of  "going  It 
alone,"  or   creating  monologues  as  opposed  to  dialogues.    But  not  all  oral 
speech  is  dialogue— as  witness  speeches  or  lengthy  oral  narratives;  and  not 
all  written  speech  is  monologue— consider  active  correspondences  or  note- 
passing  in  the  classroom.   Closed-schemas,  discourses  not  dependent  on 
interruptions  and  Interactions,  have  features  in  conwion  whether  they 
involve  oral  or  written  mediums  that  distinguish  them  from  open-schemas, 
which  have  their  own  features,  present  for  both  oral  and  written  speech. 
Nonetheless,  it  is  probably  true  that  we  learn  open-schema  strategies  first 
because  we  learn  oral  speech  first  in  its  most  coninon  form,  the  oral  dyad. 
Only  later  do  we  learn  to  make  monologues,  these  becoming  increasingly 
decontextualized  or  independent  of  shared  speaker/listener  context  for  their 
interpretation  as  we  master  the  closed-schema  strategies  for  supplying 

context  within  text. 

Like    pre-schema  strategies,  the  open-schema  strategies  may  be  real, 
as  when  I  write  a  real  letter  I  intend  to  mail,  or  simulated,  as  when  a  writer 
tells  a  story  in  a  letter  or  series  of  letters.   The  short  story  anthology 
Points  of  View  (Moffett  &  McElheny,  19 6Q  containing  examples  of  simulated 
pre-schema  discourse  ("interior  monologue"  and  Journal  entries)  and  open-schema 
("dramatic  monologue"  and  letters),  was  widely  used  in  English  composition 
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classes  at  Drake  during  the  assessment  years,  a  fact  which  may  account  for 
the  appearance  of  these  simulated  open-schemas  1n  the  sample.    In  most  cases, 
the  elements  of  open-schema  strategies    (especially  phatic  comments. a Imed 
at    establishing  a  relationship  between  reader  and  writer  rather  than  at 
cofmunlcating  content:  -HIl  How  have  you  been?"    "Well,  you're  not  going  to 
believe  this.-    "I  know  you'll  think  I'm  crazy.-  )  are  used  to  frame  clear 
whole  pieces  of  discourse  using  easily  classified  closed-schema  strategies. 
In  which  case  the  writing  was  classified  using  the  typology.    In  some  cases, 
however,  the  student  created  such  a  strong  sense  that  writer  and  reader  were 
e,gaged  In  a  dyad  that  the  closed-schema  typology  could  not  apply-the  papers 

belonged  on  another  map. 

In  drawing  the  lines  of  development  for  Figure  4.  I  have  attempted  to 
Show  that  expert  use  of  any  strategy  Is  as  cognltlvely  advanced  as  expert  use 
of  any  other  strategy,  and  that  writers  may  reach  expert  status  for  different 
strategies  1n  almost  any  erder.  maybe  becoming  expert  In  only  one  strategy 
while  remaining  novice  at  all  others.    The  direction  of  development,  from 
left  to  right  across  the   chart.  Is  meant  to  suggest  the  likely  order  of 
initially  acquiring  the  different  strategies,  given  schooling  In  our  society, 
and  perhaps  given  the  nature  of  cognitive  development  and  the  cognitive  demands 
of  the  respective  strategies.    For  Instance,  some  aspects  of  connentative 
strategy  appear  to  rely  on  a  student's  having  entered   Plagefs  stage  of 
-fomal  operations.-   Moffet  (1968)  describes  the  same  direction  of  development, 
saying  that  young  children  write  sustained  concrete  discourse  (stories)  with 
an  occasional  abstraction,  and  that  only  older  children  appear  able  to  write 
sustained  abstract  discourse,  learning  to  imbed  the  concrete  references. 

I  anticipate  a  problem  for  evaluators  when  faced  with  the  texts  of 
children  who  are  In  transition   between  the  points  represented  on  the  chart: 
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either  because  they  are  attempting  to  imitate  a  strategy  they  do  not 
fully  own,  having  Just  begun  to  learn  it  in  school  or  having  encountered 
it  in  reading,  and  do  not  yet  have  need  of,  not  understanding  the  function 
for  which  the  strategy  was  developed;  or  because  they  have  encountered  a 
new  function—for  instance,  the  need  to  express  an  idea—which  their 
current  strategy  seems  inadequate  to  deal  with.    This  latter  dllenma  may 
drive  a  writer  back  into  pre-schema  strategy  in  a  search  for  a  language 
adequate  to  the  new  challenge.    For  evaluators  concerned  with  documenting 
growth  using  improved  scores  on  writing  tests  the  problem  is  compounded 
when  a  student  develops  a  high  level  of  expertise  in  using  one  strategy, 
which  he  uses  successfully  in  earlier  writing,  but  later  attempts  to  use 
another  strategy  at  which  he  is  a  novice. 

The  writer  has  at  least  two  other  kinds  of  discourse  options  not 
represented  in  Figures  2-4,  both   of  which  may  represent  more  cognitively 
complex  task  constructions  than  so  far  described  because  they  involve 
re-combining  elements  of  function  and  structure  in  less  obvious  ways. 
The  first,  already  mentioned,  are  the  cross-combinations  of  function  and 
strategy  made  possible  by  separating  these  two  aspects  of  composing. 

A  peculiar  feature  of  the  typology  is  that  cross-combinations  of  function 
and  strategy  appear  to  be  uni-directional :  that  i$»  more  concrete  strategies 
can  be  used  to  realize  more  abstract  functions,  but  more  abstract  strategies 
cannot  be  used  to  realize  more  concrete  functions.    Figure  2  divides  the 
discourse  schemas  that  result  from  combinations  of  function  and  strategy  into 
primary  and  secondary  discourse  types  and  provides  the  combining  rule.  It 
will  be  obvious  to  most  readers  that,  while  Function  I  Is  bound  to  narrative 
strategy,  narrative  strategy  Is  not  limited  to  the  function  of  telling  a  story: 
a  reader  may  complete  a  narrative  realizing  he  has  been  preached  to— that  what  was 
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at  stake  was  an  idea,  not  a  set  of  events,  a  plot,  a  character.    The  possibility 
of  using  narrative  to  do  more  than  tell  a  story  may  be  at  the  heart  cf  the 
difference  between  good  second-rank  entertainment  novels  in  various 
genre  and  novels  that  compete  as  serious  literature-    This  has  nothing  to  do 
with  what  "lasts."    Some  good  stories  will  last  just  because  they  are  good 
stories.    Other  works  will  last  because  of  something  they  reveal  about  the 
human  condition,  rather  than  because  of  a  suspenseful  plot  and  fast  action. 

One  may  argue,  of  course,  that  really  good  literature  functions  both 
as  good  story  and  as  vehicle  for  an  idea,  which  argument  introduces  the  second 
kind  of  re-combining  of  elements  in  more  complex  task  constructions. 

I  have  found  in  the  sample,  as  i^n  the  real  world  of  adult  writing, 
clearly  distinct  types  of  writing  which  seem  to  me  to  be  best  described 
as  combining  two  functions,  and/or  two  or  more  strategies.    These  mixes 
are  not  necessarily  "mixed-up"i  although        inexperienced  writers  seem  to 
lose  track  of  function  or  shift  strategy  mid-way,  producing  mixed-up  pieces, 
better  writers  seem  able  in  many  cases  to  produce  controlled  combinations 
Identifiable  as  discourse  schemes  distinct  from  any  of'the  pure  types. 
Some  examples  will  clarify  the  possible  permutations  of  the  system: 

I  Function:  tell  a  story;  with  Stratgy  A;  narrative:  One  student 
wrote  about  the  adventures  of  "Freddy  the  Fish"  in  response  to  the  '75  topic; 
a  picaresque  tale  with  no  apparent  point,  and  no  attempt  to  reveal  why  the 
writer  might  have  wanted  to  share  this  particular  life.    Another  student 
wrote  what  might  have  been  a  good  episode  for  "Mission  Impossible."  Both 
used  narrative  strategy  to  tell  a  story,  nothing  more^ 

I-III  Function:  tell  a  storv  and  reveal  an  idea;  with  Strategy  A: 
narrative.     A  remarkable  short  story  captures  the  moment  of  truth  in  the 
life  of  a  young  terrorist,  who,  feeling  conscience-stricken  in  the  act  of 
setting  charges  to  blow  up  a  bridge,  likens  himself  to  the  carnivores,  who 
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by  preying  on  deer  and  elk    thin  the  herd  and  keep  it  healthy.    This  came 
in    response  to  the  1974  topic,  "choose  to  be  something  other  than  a 
human  being":the  student  explores  in  his  story  the  problem  of  why  anyone 
might  choose  to  be,  or  to  see  himself  as,  other  than  human. 

I-III  Function:  tell  a  story  and  reveal  an  idea-  with  Strategy  A-C: 
narrative  and  commentary.    This  class  of  writing  is  familiar  as  the 
autobiographical  essay,  or  reminiscence,  memoir,  etc.  which  has  before 
been  inadequately  characterized  and  confusingly  classified  by  systems 
which  distinguished  narrative  from  expository  writing  and  then  had  to 
decide  where  these  pieces  belonged.    This  kind  of  writing  is  widely  featured 
in  popular  magazines,  essay  anthologies  for  Freshman  composition  courses, 
and  as  classroom  assignments,  from  the  first  time  a  teacher  says,  "Write 
about  your  summer  vacation."    Unlike  good  story-telling  which  requires  that  the 
cormentary  serve  the  narration    ,  or  good  exposition,  which  requires  that 
narration  be  subordinated  to  coirinentary ,  the  personal  experience  essay 
requires  a  balance  of  both  strategies.    It  is  organized  and  functions  as 
a  story,  but  it  is  the  story  of  an  idea—a  realization,  a  discovery,  an 
t)sight  the  writer  had  as  a  result  of  a  series  of  events.    The  discussion  of 
the  ideas^-what  I  understood  before,  what  I  understand  now--is  as  important 
as  the  relating  of  events,  but  is  itself  couched  in  a  narrative,  or  at  least 
a  chronological  framework.     In  an  effectively  fused  piece,  a  reade-  might 
find  it  difficult  on  concluding  to  decide  whether  she  had  just  read  an  essay 
or  a  short  story-simply  because  both  those  terms  must  be  used  so  loosely  to 
apply  to  this  kind  of  contemplative  autobiography. 

Ill  Function:  reveal  an  idea;  with  Strategy  C.  This  kind  of  text  was 
cormon  in  response  to  the  1976  topic,  naturally,  and  represents  traditional 
notions  of  formal  argument  or  exposition.    A  thesis  is  set  forth,  and  is 
followed  by  a  series  of  coimientary  propositions,  with  any  descriptive  or 
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narrative  material  introduced    as  elaboration  or  support,  a  kind  of 
backgrounding  for  these  core  comments  which  make  up  the  writer's  argument. 

Ill  Function:  express  an  idea;  with  Strategy  A.    There  was  only  one 
clear  case  of  this  in  the  sample,  but  prototypes  in  literature  would  include 
morality  tales  and  allegories,  where  the  concrete  events  in  the  story 
represent  what  are  believed  to  be  general  truths*  not  real  occurrences. 

Combinations  of  Function  II  and  Strategy  B  with  other  functions  and 
strategies  will  be  fully  described  in  the  report  on  the  Drake  data.  At  first 
count  after  coding,  it  appears  that  the  typology  was  able  to  account  for 
better  than  90*  of  papers  in  the  sample,  with  the  other  lOX  falling  into 
pre-schema  or  open-schema  categories.    The  primary  types,  with  matched 
function  and  strategy,  appeared  to  be  marginally  preferred  solutions  for 
every  topic/year,  although  which  primary  type  was  preferred  sometimes  differed 
within  topic,  depending  on  grade  level.  Almost  every  kind  of  mix  listed  in 
Figure  2  as  possible  was  included  in  the  sample,  with  some  mixes  causing  more 
trouble  in  coding  than  others.    No  attempt  was  made  in  the  early  stages  of 
coding  to  distinguish  between  "mixed-up"  papers  and  apparently  controlled 
combinations;  it  is  expected  that  different  kinds  of  mixes  can  be  coded  in 
the  future  to  provide  additional  information  about  a  writer's  level  of  control 
over  his  or  her  chosen  strategy  or  discourse  type. 
CHOOSING  A  STRATEGY 

In  some  cases,  the  success  of  a  piece  of  writing  in  which  a  concrete 
ttrategy  fulfills  in  abstract  function  may  be  accidental.   That  is,  the  writer 
may  be  unaware  of  the  rich  and  complex  meanings  readers  are  able  to  infer  from 
his  text.    In  the  Drake  sample,  however,  successful  crossing  of  strategy  and 
function  are  generally  regarded  as  intentional  and  are  seen  to  represent  a 
highly  complex  task  construction,  available  only  to  more  experienced  composers. 
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Similarly,  a  successful  mix  of  two  functions  or  two  strategies  tends  to 
result  in  a  text  that  is  recognizably  like  a  prototype  in  adult  prose,  so  that 
the  writer  can  be  credited  with  having  attempted  a  more  complex  task,  the 
creation  of  a  secondary  discourse  type.    But  when  one  of  these  conventional 
kinds  of  mix  is  not  in  evidence,  it  is  not  clear  whether  the  writer  was 
attempting  such  a  mix  or  was  merely  shifting  ground^  attempting  to  cope  with 
topic  constraints  she  felt  were  conflicting,  or  to  cover  weaknesses  in  her 
on\  repetoire  of  strategies. 

Empirical  research  is  needed  to  help  discover  why  a  student  construes  a 
task  as  he  does:  what  is  the  range  of  options  he  thinks  he  has?  How  does  he 
want  his  text  to  function  and  why  did  he  set  that  goal?   What  strategies  will 
he  choose  to  reach  his  goal?    Is  he  an  oldtimer  or  a  newcomer  to  his  chQj|en 
strategy?   What  awarenesses  of  text  grairmar  appear  to  influence  his  whole 
tei^t  planning  or  his  sentence  level  planning?   How  Is  the  onset  of  text 
level  planning  related  to  increasing  awareness  of  discourse  schemas?  The 
discourse  typology,  refined  and  clarified,  may  be  of  some  use  in  exploring 
these  questions,  questions  which  may  help  us  discover  why  and  how  students 
change  the  difficulty  of  what  they  attempt,  even  when  a  tester  may  intend 
task  demands  to  be  parallel •    These  questions  may  also  help  us  define  task 
complexity  in  new  ways,  allowing  more  precise  evaluation  of  changes  or  growth 
in  composing  ability. 

While  discourse  functions  may  be  recognized  on  a  deep,  fairly  unconscious 
level,  I  believe  that  strategies  can  be  made  conscious,  learned  deliberately, 
and  applied  selectively.    On  the  other  hand,  experience  with  writers  like  those 
in  the  sample  suggests  that  students,  in  selecting  a  strategy,  are  generally 
working  from  a  confusing  array  of  instincts  and  inhibitions  which  have  little 
to  do  with  the  appropriateness  of  the  strategy  to   the  desired  function. 
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Table  2  outlines  some  factors  I  think  may  influence  a  student's  choice 
of  strategy »  whether  pre-schema  or  schema-based.    Future  research  will 
determine  whether  these  are  real  and  how  they  may  operate  for  given  students. 
I  may  assert^  logically,  however,  that  availability  is  absolutely  determining. 
A  writer  cannot  use  a  strategy  which  is  not  available  to  him»   Given  the 
availability  of  more  than  one  schema-based  strategy,  together  with  the 
universal  availability  of  the  strategy  I  call  pre-schema,  any  or  all  of  the 
remaining  factors  may  come  into  play  in  any  combination.    I  suspect  that  some 
of  these  create  conflicts  for  the  writer  and  that  keeping  these  factors  in 
mind  will  help   researchers  understand  papers  whose  task  constructions  are 
not  easily  classifiable  by  the  typology. 

It  is  premature  to  Judge  how  well  this  typology  will  assist  efforts 

to  define  task  complexity  in  writing  or  to  assess  a  student  writer's  level 

of  development  on  the  basis  of  texts  produced  for  evaluation.    So  far, 

informal  sharing  with  teachers  of  composition  suggests  that  the  typology 

can  have  a  clarifying  effect  of  distinguishing  among  types  of  writing  tasks 

set  by  or  for  students  and  can  help  teachers  Identify  some  of  the  strategies 

students  may  have  only  partially  mastered  or  may  be  moving  between.  A 

teacher  of  high  school  Juniors  Identified  two  kinds  of  text  typical  of 

students  in  Intermediate  stages  between  mastery  of  n?Trative  and  mastery  of 

commcntative  strategy.     She  submitted  these  two  samples: 

Assignment:    It  is  said  that  we  learn  best  from  our  mistakes.  Agree 
or  diaagree,  drawing  on  examples  from  your  own  experience* 

Stage  One,  student  response:  "It  is  said  that  we  learn  best  from  our 
mistakes.    I  know  of  one  time  when  I  learned  an  important 
lesson  by  doing  something  stupid.    I  was  twelve  at  the  time... 
(narrates  gy^nt:  four  pages  of  lively  story- telling.) 
...I  thought  iny  mother  was  paranoid,  but  now  I  know  1*11 
be  telling  my  kids,  "You  shouldn't  trust  strangers." 
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Staoe  Two.  student  response:  Mt  Is  said  that  we  learn  best  from 
"'^ur  Si  tates.   That'ls  probably  true  because  unti   we  make  a 
mistake  we  don't  have  iny  reason  to  want  to  know  something. 
T'h    'learned  I  lot  of  'things  by  jlsta  es   .n  mos 

of  these  have  stayed  with  me  more  than  ^^e  tMnj^JJ'J^^^.t 
by  studying  courses  In  school  or  »^«d1ng  or  listening  to  what 
mv  oarents  or  other  people  told  me.   Maybe  those  things  stuck 
7n  w        KcSuse  1t's%o  humiliating  to  make  a  mis ta  e 
You  can't  forget  the  embarrassment,  and  you'd  rather  risk  deam 

i:}t's'5Sod*to'"r;member  that  our  mistakes  teach  us  something 
and  that  we  don't  have  to  be  ashamed  to  do  sjJ'J^'  "9  ^^^J' 
•cD#c1allv  when  we  didn't  know  any  better.   That  1$  how  we 
leSrn.    B^lng  Jf^ald  of  making  mistakes  can  keep  you  from 
learning." 

The  first  student  Is  still  largely  committed  to  narrative  strategy: 
her  introduction  and  conclusion  are  concessions  to  the  expository  nature 
of  the  prompt,  but  they  are  minimal.  Interfering  as  little  as  possible  with 
the  story.    Students  less  aware  of  essay  requirements  often  Ignore  even 
these  concessions,  beginning.  "It  happened  when  1  was  twelve.,,' 

or.  -One  time,  when  I  was  twelv  "A  teacher  wanting  to  lead  a  stage 

one  writer  further  Into  the  language  of  Ideas  might  ask  the  student 
simply  to  expand  the  conr«nttry  of  the  opening  and/or  the  closing,  or  to 
write  a  second  narrative  of  another  time  when  she  learned  from  a  mistake, 
then  to  write  a  paragraph  co^arlng  or  contrasting  the  two  experiences.  Cawing 
some  conclusions  about  how  or  why  people  learn  from  mistakes. 

The  second  student  has  fully  abandoned  narrative  strategy,  with  a  common 
result:  he  Is  able  to  produce  only  two  paragraphs.   These  paragraphs  are 
rich  in  Insights  worth  exploring  In  an  essay.    Further,  they  present  a 
paradox  which  will  require  mrt  thinking  by  the  student  If  he  Is  to  untangle 
It:  the  first  paragraph  suggests  that  being  afraid  of  making  mistakes,  or 
being  ashan«d.  Is  why  we  l.arn  from  our  mistakes;  the  second  suggests  that 
being  afraid  of  making  mistakes  may  keep  us  from  learning.   Two  good  Ideas; 
the  paradox  Is  only  apparent.    But  the  paradox  Is.  apparent,  rather  than  hidden. 

ERIC  ^16 


because  the  student  risked  talking  about  Ideas,  struggling  to  make  explicit 
understandings  which  were  only  intuitive  before.    The  struggle  is  essential 
if  the  student  is  to  acquire  that  all-purpose  academic  discourse  schetne; 
Function  III,  Strategy  C,  expressing  an  idea  using  conrnentary    as  the  core 
of  the  text,  working  out  the  logical  relations  between  the  parts  of  the  idea. 
But  the  stage  two  student  is  often  greeted  with  such  negative  evaluations 
he  may  be  excused  for  wanting  to  retreat  to  the  old  successes  of  stage  one 
or  even  pure  narrative.    "This  is  too  short.    You  contradict  yourself. 
You  don't  have  a  single  conrete  example  from  your  own  experience.  You  shift 
pointjof  view,  from  first  to  second  person;  and  what  is  this?  the  royal  'we*? 
As  the  student   says,  "Being  afraid  of  making  mistakes  can  keep  you  from 
learning." 

Neither  of  these  essays  would  receive  a  top  score  during  a  holistic 
assessment  In  which  at  least  some  students  managed  to  produce  Strategy  C 
essays  of  better  than  novice  quality,  essays  which  included  a  full  range 
of  propositions  from  abstract  to  concrete,  with  concrete  description  or 
narration  carefully  attached  to  corinentary  statements,  which  in  turn  were 
arranged  to  form  a  coherent  core  argument.    But  it  is  highly  likely  that 
raters  would  prefer  the  stage  one  paper  with  its  expert  use  of  narrative 
strategy  to  the  stage  two  paper  of  the  novice  commentator  with  its  all  too 
evident  flaws.    No  provision  could  be  made  for  recording  that  the  stage  two 
writer  may  have  moved  closer  to  the  goal  than  the  stage  one  writer  if  one 
compared  their  respective  task  constructions. 

Much  greater  clarification  of  discourse  features,  and  of  the  role 
played  by  text  level  expectations  of  both  writers  and  readers,  seems  needed 
before  it  will  be  possible  meaningfully  to  measure  improvement  in  composing 
ibilities. 


NOTES 


The  papers  in  the  sample  were  pooled  across  all  topics,  all  years,  in 
a  single  holistic  scoring,  to  obtain  scores  that  allowed  comparison  of 
all  papers  according  to  a  general  impression  of  quality.    Details  of 
this  scoring  will  be  available  in  the  final  report  on  this  research, 
from  the  NIE  Writing  Assessment  Project,  Marcia  Farr,  NIE  Project  Officer, 
Reading  and  Language  Studies,  Grant  No.  NIE-G-80-0034,  c/o  the  Bay  Area 
Writing  Project,  University  of  California,  Berkeley.    Working  title: 
Technical  Report  No.  5,  The  Drake  Longitudinal  Study. 

^    It  is  important  not  to  confuse  task^cpmplexi ty  wi th  text  complexi ty_,_ 
Joe  Williams  makes  this  criticalTistinction  11979) ,  pointing  out  that 
very  complex  cognitive  activity  may  be  required  to  produce  a  text  that 
is  simple  and  direct  from  the  reader's  point  of  view,  while  a  very 
complex  text  may  be  produced  by  the  simple  expedient  of  transcribing 
thoughts  as  they  occur  to  the  writer. 

3    Kinneavy  (1979)  describes  the  aims  of  discourse  in  a  volume  which 
exhaustively  analyzes  examples  of  each  type,  attempting  to  establish 
stylistic  features  these  texts  have  in  common,  while  reserving  his 
discussion  of  the  "modes"  of  discourse  to  a  later  volume,  not  yet 
prirvted.    It  is  with  some  he«;itation  that  1  offer  yet  another  discourse 
classification,  rather  than  waiting  for  his  definitive  second  work, 
but  I  find  his  system  useless  for  the  student  sample  in  question,  as 
well  as  inadequate  to  account  for  many  text  types  I  have  encountered 
in  adult  letters,  both  in  and  out  of  academia.    The  separability  of 
function  and  strategy  suggests  the  need  to  classify  each  text  on  both 
counts  in  order  to  describe  its  essential  differences  from  other 
texts,  rather  than  first  classifying  all  texts  according  to  function 
or  aim,  as  Kinneavy  appears  to  do,  reserving  til  later  an  attempt  to 
describe  their  modes.    As  can  be  seen  in  the  application  of  my  typology, 
Function  A  in  combination  with  Strategy  A  results  in  a  very  different 
kind  of  text  from  a  text  which  combines  Function  C  with  Strategy  A, 
and  this  difference  establishes  a  different  class  of  text  rather  than 
ore  being  a  sub-species  of  the  other. 
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T«b1e  1 

Topics  Used  1n  Six  Years  of  Dr»ke  Writing  Assessment 


1973  Write  ibout  •!»  event  you  wl»h  you  h«d  wltneiied  or  could  wltne«».    The  event 
can  be  reel  or  Huiglned;  the  tine  of  the  event  can  be  P«»t.  pretent  or 
future.    «*ke  it  clear  why  the  event  la  alinificent  to  you.    You  may  vrlte 

a  jourrial  entry,  letter,  dialogue,  ■onolojue. -eieay,  atory,  autobiography, 
or  other  form. 

1974  If  you  had  to  choose  to  be  ao«ethin«  other  than  a  human  being,  what  plant  or 
animal  or  other  for.  would  you  chooae?    In  your  writing,  give  your  reader  .or^e 
Idea  of  what  you  think  It  would  be  like  to  be  that  form,  and  of  why  you  chose 
it.    You  .ay  do  thio  writing  a*  a  Journal  entry,  a  letter,  a  dialogue,  a  acory, 
an  autobiography,  an  caaay.  ■  poem.  etc. 

1975  If  you  could  change  placea  with  aoMone  elae.  who  would  It  be?    The  person  you 
write  about  can  be  Uvlng,  dead,  drawn  fro-  p.at  or  preaent.  from  books,  films, 
ate      or  from  your  own  Imagination.    In  your  writing  give  your  reader  aome 
Idea'of  what  It  would  be  like  to  be  that  peraon,  and  of  why  that  life  appeals 
to  TOU.    You  may  do  thla  writing  aa  a  Journal  entry,  character  aketch, 
dialogue,  letter,  etory.  autobiographical  eaaay.  argu.ent,  poem,  or  other  lorr. 

1976  Not  all  inventlona  have  been  good  for  all  hu«nlty.  Name  ""'Z";;*"^ ^ 
vould  be  better  off  without,  and  make  it  clear  why.  You  .ay  do  this  writing 
aa  an  essay.  Journal,  letter,  story,  or  other  form. 

1977  Imagine  that  a  .«U  «roup  of  people  will  be  aent  to  •  P^;'^*^' 
Food,  Clothing,  ahelter  and  transportation  have  been  provided  for.  Ycu 
are  ^K>ng  thoae  aaked  to  aelect  a  few  additional  tMnga  to  be  aent  along 

in  the  limited  apace  available  In  the  ahip.    What  one  item  would  you 
recoMaend.  and  why?    Tou  m»y  write  your  reco— endatlon  in  the  form  ol  ■ 
atory,  a  dialogue,  a  letter,  a  apeech,  an  saaay  or  other  for.. 

1978  Write  about  ao-e  way  in  which  your  life  haa  been,  or  raight  be  Influenced. 
Tou  .ight  write  about  the  Influence  of  Mother  peraon,  •  book  or  film, 

an  idea,  or  an  event  iuch  aa  a  triumph  or  defeat,  or  a  audden  gain  or  loss. 
Make  it  clear  Juat  what  or  who  Influenced  you,  and  what  the  effect  was  upon 
TOU,  or  what  the  effect  could  be  u^on  you.    Tou  -y  do  thla  wrltlnB  as 
.  Journal  entry,  character  eketch,  dialogue,  letter,  atory,  autobiographical 
aaaay,  or  other  form. 
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Table  2.  CHOOSING  A  STRATEGY 


A  list  of  fictors  wMch  My  detenrlne  or  iffect  a  student's 
choice  of  striteay*  whether  pre-schemt  or  scheim-bised. 

1.  AVAIIWIIITY.  (Whtch  itntegles  4oes  the  student  have  in  repetolre?) 

2  rHEFIREWCE.  (Does  the  student  prefer  some  strategies  because  of 

famniarlty,  facility,  rtQardless  of  purpose  or  context 
of  the  writing  task?) 

3  APPROPRIATENESS  TO  COKTin^  (Does  the  student  have  Ideas  about  what  strategies 

are  appropriate  to  writing  tests,  rtgardless  of  topic, 
•r  the  Ippcrtance  of  applying  recently  learned  strategies 
to  deiwnstrate  MStery»  rtgardltss  of  their  naturalness  for 
•  given  purpose  or  asslgniMnt?) 

4.  APPROPRIATENESS  TO  WYEN  TASK.  (CerUin  topics  suggest  eipHcltly.  or  tend  to 
elicit  certain  strategies,  by  eliciting  certain  purposes 
wMch  art  MSt  easily  Kcoaipllshed  with  Batching  strategies, 
e.g.,  tell  t  story  Is  onl^  accomplished  by  narrative.) 

5  APPROPRIATENESS  TO  WRITER'S  CHOSEN  WRROSE.    (If  writer  chooses  a  purpose  other 

than  the  obvious  one  suggested  by  the  topic,  he  aiay  choose 
a  strategy  appropriate  to  his  own  purpose  rather  than  one 
Woorlate  to  the  enpected  purpose  suggested  by  the  topic.) 

6  IDIOSYNtliATIC  OR  HIGHER  ORDER  CRITtRIA.  (The  writer  «ay  have  personal 

criteria,  such  as  wishing  U  be  novel  or  unusual ,  which 
My  Influence  him  to  choose  an  uncoM»n  solution: 
for  Instance,  deliberately  nlsMtchlng  purpose  and 
strategy  because  he  does  not  wish  to  do  the  obvious,  or 
because  he  believes  he  tan  achieve  his  purpose  Mre 
effectively,  perceiving  a  complex  Interaction  between  form 
and  function  which  will  coMwnlcate  Mre  than  he  •Ight 
should  he  use  expected  strategies  for  a  given  purpose.) 


50A  ' 


ERIC 


1973             1974  1975 

197^ 

1977 

1978 

 ' 

FRESHMAN 

1  Cohort  I 

:  30 

jCohort  II 
l.^ylQ  

30  papers 

30  papers 

SOPHOMORE 

30 
papers 

Cohort  I 
30 

bohort  III 
1  30 

1  30  papers 

30  paper?^ 

JUNIOR 

30  papers 

30  pipers 

Cohort  I 
30  ' 

Cohort  II 

I  Cohort  III 
V    30  1 

'  30  papers 

SENIOR         j  30  papers 

30  papers 

1 

30  paoers 

! 

Cohort  1  1 

30  1 

1  Cdtort  n 

I  ^  . 

1  Cohort  III 
1  3^ 

Totil  p»p«r*-  720 

Three  lonsitudinal  »«tpl«t-  360  ptpert 
SupplemenCiry  gtapllng  -  360 

Figure  1. 


30  students  30  stucJents  30  studentt 

12C  pipers    120  papers  120  papers 

Cohort  I,     Cohort  II,  Cohort  III, 

class  of  76  class  of  77  Cl«««  of  78 

S?Mple  Population 
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Figure  2. 

A  Discourse  Typology;  describing  the  task  constructions 
founa  in  student  prose  written  In  response  to  6  prompts. 


Discourse  Function 


Discourse  Focus  Discourse  Strategy 


I 

t 


J. 

TELL  A  STORY 


Change  over  Time 


A. 

narration 


II. 

REVEAL  AN  ENTITY 


Identity  over  Time 


description 


V 

4-» 
U 

m 

L 


III. 

EXPRESS  AN  IDEA 


Relations  beyond  Time  coBwentary 


Discourse  types  ,  resulting  from  contlnatlons  of  function  and  strategy 
found  in  the  senile  or  in  adult  literature: 

1.  Primary  types,  matched  function/strategy:  lA;  IIB;  IIIC. 

2.  Primary  types,  mixed  functlon/straUgy:  llA;  ♦  ;  IIIA;  IIIB. 

3.  Secondary  j^J^^'^"*^^^^  function/strattgy:  l-III/A-C;  1-Il/A-B; 

4.  Secondary  types,  mixed  functlpn/st'-attgy;  MI/A;  Il/A-B; 

1-IIl/A;  l-III/A-Bi  IMll/A;  ll-IIl/A-B;  Ilrlll/A-C;  etc. 

♦Coitining  rule  (empirically  derived)    aiiMs  mrt  concrete  strategies 
to  serve  more  abstract  functions,  but  concrete  functions  cannot  be 
served  by  more  abstract  straUgles;  hence.  Function  I  can  only  be 
realized  by  Strategy  A;  Function  II  by  straUgies  A  and  B  (ideally); 
and  Function  HI  by  any  strategy.  A.  B,  or  C.  By  this  rule  there  should 

be  no  IB.  or  IC  (true,  in  fact);  or  II/C.  or  Il-Ill/C  (these  latter  occur. 
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Flqurt  3, 

Proposition  Structurt  of  Thret  Dlscourst  Strategies: 


NAKMTIVE 

I 


ttSCtinON  COttlEHTARY 
(Cj) 


I 

etc 


etc. 


E     •   Event  stitewnts:    "X  happened/       1$  happening."   "X  did.,." 

.  Descriptive  $tatei«nts:   "X  is..."   "X  does...*   "X  happens../  (habitual  aspect 
Descriptive  swwpenu.  ^„  predicate;  subject  of  vtit  refers  to 

actual  entity  or  action  treated  as  entity,  not  to  hypothetical 
construct.) 


C     •   Coamentatlve  statements: 

(jilso7descr1pt1vrsta'teJi»n^^^        'X  i%.V.'  become  coementatlve 
when  subject  or  subject  co«pleeient$  are  hypothetical  constfDcts.J 
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rigyrt  4,  NypothesliH  Chronology  of  the  Acquisition 
and  OcvelofMent  of  Writing  Strategies. 


tngaged  Pre-Schewa 
strategies 


i->  Slmilatcd  Pre-Schi«a  Strategies 


Open^Sc^eaia 


Stratefles; 


Engaged  Dyadic  Strategies - 
(novice) 


(anprtntlce)- 


Closeg^Schewa  Strategies; 
Narratl««_y. 

(novice) 


(ai^ 


(apprentlce)a 


i^Detcrlpitlve  J 


(novice) 
I 

(apprmtlce)* 


i^SliwIated  Dyadic  Strategies 

(novlce>^appr«ntlct)-«>-- ^eipert) 


i^(eiiptrt) 


=>GoMentatlve 


(n«v1ce) 
a 


(a9^tlct)««-«i  M^eapevt) 
—  —  —  —  —  —  — —  ►(eiptrt) 


-•^enptrt) 


hypothesized  direction  of  developnant.  Ntwaan  schawa  types* 

1^  hypothesized  direction  of  developaMt  within  schema  types.  ^*  '^•^ 

^  between  strategy  types 

logically  reqvlrad  direction  of  develaptnt  within  strategy,  nhlch  nay  In  actuality  be 
broken  by  stops  and  reversals  along  the  tiay 

rr  o  * 
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Chapter  5:    Part  2 


BEYOND  HOLISTIC  SCORING:  RHETORICAL  FLAWS 
THAT   SIGNAL  ADVANCE  IN  DEVELOPING  WRITERS 


Ellssa  Harantz  and  Catharine  Keech 
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The  study  analyzes  a  longitudinal  sample  of  student 
exams  collected  across  four  years  from  90  high  school  stu- 
dents who  participated  In  annual  »*rltlng  assessments.  The 
writing  test  each  year  consisted  of  an  hour-long  composition 
written  to  a  prompt  which  allowed  the  students  to  draw  on 
personal  experience  or  imagination  and  to  choose  their  mode 
If  discourse.     All  of  the  essays  were  »«1  ^  ^r'^dS 

in  a  single  pooled  sample  for  purposes  of  this  study. 
Holistic  scores  represent*  single-Impression  rapid  estimates 
of  over-all  rhetorical  effectiveness  of  an  essay  compared  to 
other  essays  In  the  sample.  A  conspicuous  problem  recurring 
consistently  In  this  widely  used  assessment  procedure  is  the 
instability  of  student  performance  from  one  essay  test  to 
another,  as  exemplified  by  low  test/re-test  ^^J^^^'^J 
failure  of  Individuals  and  groups  to  produce  consistent 
Improvement  on  these  essay  tests  from  year  to  year  even  with 
concentrated  writing  Instruction,  and  by  the  ^!;«2^«"*^V  jlth 
which  Individual  or  group  mean  scores  actually  decline  on 
post  tests  or  second  trials. 

Attempts  to  solve  this    measurement    problem  In 
testing    have    focused    on  developing  "ore  precisely  <leflned 
topics  and  on  more  reliable    rating    procedures,    but  there 
have    been    no    sustained    efforts    to  expl ain  what  actual ly 
changes  In  student  writing  across    several  test 
This    study    Investigates  the  longitudinal  dimension  of  this 
phenomenon  by    following    student    performance    "cross  four 
Snnual     test     occasions.    The  study  shews  that  some  students 
attain  lower  holistic  ratings  on  later  test    essays  because 
they    use    more    complex    or    difficult    composing  $k  lis  or 
because    they    have    defined    the    rhetorical    task  the 
assigned    prompt  more  complexly  than  In  a  prior  test.  Since 
?he  loosely  defined  writing  prompts  offer  ample  opportunity 
for     Individual    variation     In  task  interpretation,  students 
who  have  learned  new  skills  or  who  begin  to  JJ*-*^**;;* 
tasks    more    complexly    may    attempt    more    than  they  can  do 
easily,  with  the  result  that  their  essays  are  less  '^^etorl 
cally    well-formed    than    on    earlier    occasions    when  they 
attempted  less. 

This  apparent  regression  In  performance  which  actually 
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signals  movennent  forward  in  cognitive  or  affective  develop- 
ment or  learning  has  been  termed  "d i se qu i 1 ibrat i on*  by 
Piaget  in  his  stage  frodel  of  cognitive  development 

EIuccs^  sf  the.  iiu^:y 

The  purpose  of  this  study  is  to  exaxine  the  extent  to 
which  students  atteupt  to  do  different  kinds  of  writingt 
either  spontaneously  or  in  response  to  strongly  suggestive 
cues  of  the  writinj  prompts  they  receive  in  a  series  of 
annual  hoi  1st  Ic  wr  1 1  Ing  tests  taken  during  their  ninth  to 
twelfth  grade  years  In  high  school.  It  was  hypothesized 
that  differences  In  the  student's  task  CCDSl£UCLiQDt  or  what 
the  student  attempted  to  dot  aight  help  to  account  for  unex- 
pected directions  of  change  In  student  performance  scores 
from  year  to  year.  In  other  words t  when  a  student  does  the 
same  kind  of  writing  in  1976  that  he  did  In  1975,  using 
similar  strategies  to  accomplish  similar  purposes,  an 
evaluator  might  reasonably  compare  the  student's  two  scores 
to  determine  whether  he  was  doing  the  same  thing  any  better 
after  a  year's  maturation  and  some  writing  instruction.  If, 
on  the  other  hand,  the  student  does  something  different  in 
1976,  using  new  strategies  to  accomplish  new  purposes,  his 
score  will  not  Indicate  how  much  he  has  Improved  at  doing 
the  old  task;  rather.  It  will  reflect  how  well  he  does  on  an 
altogether  new  kind  of  task.  This  research  Is  a  first  stage 
effort  to  distinguish  among  the  factors  which  contribute  to 
the  seeming  instability  of  student  performance  from  one  test 
occasion  to  another.  Ultimately,  this  research  will  direct 
attention  to  the  developmental  features  of  text  design  and 
task  analysis  which  also  Influence  quality  rating  of  student 
essays  in  seemingly  paradoxical  ways. 

The  Improvement  of  quality  ratings  of  student  perfor- 
mance on  writing  assignments  has  long  been  equated  with 
improvinj  student  writing  abnitles,  and  this  advancement  In 
quality  is  generally  accepted  as  the  goal  of  writing 
instruction.  Even  when  educators  use  multiple  choice  tests 
as  indirect  measures  of  writing  skill  (because  of  their  ease 
of  administration  and  the  superior  score  reliability  of 
these  measures)  they  recognize  that  such  tests  must  ulti- 
mately be  validated  by  relating  their  scores  to  performance 
on  direct  measures,  actual  writing  tasks  in  which  the  stu- 
dent demonstrates  his  ability  to  construct  or  compose  a 
response,  not  merely  select  a  response  (Godschalk,  1966; 
Oiederich,  1974;  Breland  and  Gaynor,  1979;  Cooper  and  Ddell, 
1979;  Spandel  and  Stiggens,  1980;  Schools  Council,  UK,  1966; 
NCTE  1975) . 

As  noted  above,  h  perplexing  problem  has  baffled  and 
frustrated  teachers,  researchers,  and  evaluators  of  writing 
instruction.  For  as  children  learn  to  write,  they  should 
perform  better:  their  scores  on  successive  essay  tests 
should  Improve.  If  children  don't  increase  their  scores,  it 
Is  easy  to  assume  that  they  have  made  no  advances  in  writing 
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abilTty.  Alternatively*  one  can  seek  other  explanations. 
One  may  examine  the  test  measures  used:  were  the  two  test 
instruments  truly  parallel  in  what  they  required  students  to 
do;  were  the  t*fO  tests  scored  in  the  same  manner?  Testers 
have  reported  Improving  score  reliabilities  substantially  by 
Improving  inter-rater  agreement*  by  phrasing  prompts  care- 
fully to  be  sure  deirands  are  parallel*  by  taking  longer 
writing  samples  to  improve  discrimination  of  the  measure* 
and  by  taking  more  than  one  writing  sairple  at  each  test  sit- 
ting (Breland  and  Gaynor,  1979;  Steel*  1979).  All  of  these 
procedures  help  reduce  the  measurement  problem  to  some 
extent;  howevert  none  examines  what  actually  changes  in  stu- 
dent writing  from  test  to  test;  therefore*  these  corrective 
measures  fail  to  reveal  a  third  likely  source  of  variation 
in  student  performance  which  has  radical  implications  for 
both  evaluating  and  teaching  cooipositicn  skills. 

To  understand  variations  in  student  writing  performance 
on  tests  it  is  important  to  distinguish  between  the 
examiner's  gi^en  tlSk"  the  set  of  constraints  and  options 
expressed  or  implied  in  the  text  of  the  writing  prompt--  and 
the  student's  construed  or  MQ5lCUCtCl3  task"  the  set  of 
constraints  actually  honored  (and  the  options  actually 
chosen)  by  the  student  in  composing.  The  student's  con- 
structed task  is  derivable  from  the  text  of  his  composition 
by  noting  the  task  constraints  honored.  While  test  makers 
do  attempt  to  control  the  given  task  tc  create  parallel  task 
demands  on  alternate  occasions*  they  are  less  able  to  con- 
trol the  student's  ccnstrual  of  the  task. 

In  other  performance  areas  like  wusic*  a  learner  is 
given  a  more  difficult  piece  to  play  or  a  new  instrument  to 
learn*  and  in  mathematics*  both  teacher  and  student  can  be 
aware  of  degrees  of  difficulty  in  problems.  But  in  composi- 
tion it  is  possible  to  give  students  identical  tasks  on  two 
different  occasions  and  have  them  construct  such  different 
composing  tasks  that  In  an  early  version  of  the  task*  a  stu- 
dent might  well  appear  expert  In  his  response  to  the  problem 
he  set  hinself;  while  In  a  later  version*  he  or  she  might 
suddenly  become  a  novice  as  he  or  she  attempts  to  solve  a 
far  more  complex  problem.  That  Ist  writing  prompts  are  usu- 
ally loosely  structured  In  the  sense  that  the  particular 
given  task  (writing  assignment)  may  be  construed  as  offering 
a  range  of  possible  real  tasks*  any  of  which  the  writer  may 
elect  to  do. 

In  an  effort  to  rar.k  writing  assignments  according  to 
difficulty  level t  It  Is  not  sufficient  Just  to  compare  the 
literal  texts  of  the  prompts*  which  ostensibly  present  a 
particular  given  or  intended  task.  Students  will  inevitably 
vary  in  how  they  interpret  and  respond  to  that  text  and  In 
how  they  construct  the  task  that  they  will  actually  attempt 
(Murphy  et  al . *  1982;  Keech*  1982).  Open-ended  assignments* 
simply  stated*  may  be  offered  again  and  again  over  all  the 
years  of  a  writer's  career  from  his  first  awkward  printings 


to  his  adult  years  proficient  composing?  the  actual  dif- 
ficulty of  the  assignmentf  howevert  will  increase  in  proper-* 
tlon  to  the  student^s  very  ability  to  define  the  task  more 
complexly.  Reasonable  ta sk  def in i t ion  It  one  of  his  most 
important  writing  skillst  and  will  develop  along  with  syn- 
taxf  sense  of  paragraphinjt  and  other  linguistic  knowledge. 
An  unreasonable  narrowing  of  topic  constraints  on  the  other 
handf  may  not  only  Inhibit  variation  In  construalt  but  it 
also  may  llnit  measurement  of  the  student's  ability  to  Jug- 
gle constraintSf  set  goalst  make  planst  select  strategies-- 
all  composing  skills  as  Important  to  measure  as  the 
student's  ability  to  transcribe  language  using  correct  writ- 
ten conventions.  In  other  wordst  determining  the  student's 
individual  construction  of  the  task  Is  of  primary  Interest 
to  evaluators  and  researchers  who  wish  to  measure  a 
student's  growth  or    Improvement  In  writing  ability. 

Mhat  Is  needed  tas  a  preliminary  to  any  effort  at 
measuring  development  of  writing  ability  Is  a  method  of  com- 
paring what  students  actually  do  and  how  they  do  itt  from 
one  occasion  to  the  next.  A  system  of  task  dfi&C^lOtlflQS  Is 
requlredf  through  which  student  texts  can  be  analyzed  to 
determine  whether  or  not  two  compositions  represent  the  same 
or  different  types  of  tasks:  which  purposes  does  the  student 
appear  to  be  almlnj  for?  which  strategies  does  he  use? 
Only  after  we  are  able  to  describe  more  exactly  what  the 
student  actually  proluces  In  response  to  test  promptst  will 
we  be  able  to  Infer  the  student's  conceptualization  or  con- 
struction of  the  writing  task*  Then  we  will  be  able  to  gen- 
erate an^  test  hypotheses  about  the  relative  level  of  diffi- 
culty of  what  Is  attempted  on  different  occasions.  That 
research  shouldt  In  turnt  make  possible  a  proper  Investiga- 
tion of  the  nature  of  developing  writing  skiUst  by  allowing 
us  to  distinguish  between  a  student's  mastery  of  old  fpirms 
long  practicedf  and  his  novice  attempts  to  try  out  new  pat- 
terns. Furthcrt  such  research  may  ultimately  allow 
researchers  to  predict  which  kinds  of  tasks  are  learned 
f irstf  which  followt  and  what  various  spontaneous  task  con- 
structions employed  by  students  may  Indicate  about  a  given 
student ' s  part  leu  la r  level  of  development . 

The  first  step  In  this  researcht  thereforet  was  to 
derive  a  typology  of  task  construction  which  would  uniquely 
characterize  the  range  and  variation  of  the  written  texts 
produced  by  students  In  the  longitudinal  sample  as  well  as 
the  kinds  of  texts  traditionally  classified  jy  discourse 
theorists  (Balnt  1890;  Kinneavyt  1979;  Brittont  19_;  Emigt 
19_;  etc.)  In  terms  of  literary  "genres.*  Once  it  was 
derivedf  the  typolojy  was  applied  to  a  sample  of  student 
paperSf  and  cross-tabulations  were  drawn  to  determine  con- 
comitant variation  In  promptt  grade  levelt  task  construc- 
tlont  quality  rating  (holistic  score).  This  study  thus 
provides      us      with    data    through    which    we    can  compare 


Individual  student  performance  (In  terms  of  task  construc- 
tion and  score)  to  the  favored  task  construction  and  mean 
score  o^  all  students  in  a  given  topic/year.  Students  ^ho 
appeared  to  deviate  in  either  or  both  dimensions  from  the 
developmental  pattern  common  to  the  larger  sample  t  as  v»eii 
as  students  who  appear  best  to  typify  the  common  jattern, 
will  be  selected  for  detailed  case  study  in  future  research. 

liifi  Q.^ti  Baifi*  *  longitudinal  sample  of  student  essays 
was  drawn  from  the  total  set  of  essays  foduced  during 
annual  writing  assessments  at  Sir  Frances  Drake  High  School, 
during  the  years  1973  through  1978.  The  sa^iple  selected  for 
the  study  consists  of  three  sub-samples  of  longitudinal  data 
and  one  sub-sample  of  cross-sectional  data,  as  shown  in  tl3Z 
UL&  I* 
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Three  cohorts  of  students  were  Identl^^leds  a  cohort  consists 
of  30  students  who  attended  Drake  High  School  together,  com- 
'     pleting  all  four  years,  fron  freshman  year    through  gradua- 
tion.     Cohort     1  attended  DHS  from  1973  through  1976  tactu- 
ally  entered  in  the  fall  of  1972)  and  participated    in  four 
soring    writing    assessments);  Cohort  II  participated  in  the 
writing  assessments  conducted  from  1974  through  1^77;  Cohort 
III    participated     In    writing    assessments     In  1975  through 
1978.    The  thirty  students  forming  each    cohort    were  drawn 
from    a  larger  pool  of  all  students  who  completed  four  years 
at  Drake  and  for  whom  test  papers  were  available.    The  orl- 
Tinal     sample    was    stratified    for  senior  performance  only, 
into  high,  middle  and  lew  scoring  writers.    Roughly  propor- 
tional    random    samples    were    drawn    from  the  three  scoring 
strata,  slightly  larger  than  the  final  sample  of  30  students 
to    be    used     In    making    the    final     cohorts.    The  selected 
seniors  were  traced     t+irough    preceding    years,    with  cases 
dropped    fron    the  cohort  If  they  did  not  produce  a  total  of 
four  papers  over  the  four  years.    With  slightly  Tiore  than  30 
cases    still     renalnlng    in    each  cohort,  it  was  possible  to 
drop  additional  cases  using    sex    as    a    criteria,    so  that 
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approximately  equal  numbers  of  males  and  females  remained  in 
the  final  cohorts.  The  resulting  sample  consisted  of  three 
longitudinal  sub-sairples »  or  cohorts  of  studentst  with  30 
students  In  each  cohort.  Each  student  had  produced  four 
essays  over  four  years  of  writing  tests. 

In  addition  to  the  longitudinal  sub-samplest  a  cross- 
sectional  sample  Mas  drawn  from  the  original  set  of  essays 
to  represent  student  performance  on  each  assessment  topict 
within  grade  levels  not  represented  In  the  longitudinal 
cohorts.  For  Instancet  the  longitudinal  sample  produced 
only  freshmen  papers  for  the  year  1973.  In  analyzing  these 
paperst  It  would  be  misleading  to  attribute  certain  features 
of  the  writing  to  the  fact  that  the  writers  were  freshmen  if 
we  did  not  also  have  upper-class  student  writing  to  check 
whether  these  features  were  particular  to  responses  to  the 
1973  toplct  rather  than  simply  a  result  of  freshman  writing 
tendencies.  The  cross-sectional  "simulated  cohorts*  provide 
a  normative  vatrix  for  toplcst  as  well  as  writing  produced 
by  any  given  grade  level  In  response  to  several  topics. 

A  different  writing  prompt  was  offered  each  year;  topic 
and  year  may  then  be  combined  In  a  single  variable  called 
topic/year t  Indicating  that  Identifying  a  particular  yeart 
e.g.t  1973t  also  Identifies  a  particular  topic  which  was  not 
repeated . 

In  the  six  assessment  prompts  used  at  Drakct  the  writ- 
ing instructions  reirained  fairly  similar  from  year  to  yeart 
allowing  students  to  choose  among  a  variety  of  genret  such 
as  storyt  essay t  ■  journal t  etc.  What  changed  from  year  to 
yer  was  the  iQclCt  or  content  focus.  Thust  patterns  of 
differences  In  student  responses  from  year  to  yer  appear  to 
be  a  result  of  differences  In  the  way  the  test  question 
described  the  actual  toalcs  of  the  writing,  rather  than 
differences  In  Instructions  to  the  writer  about  how  to 
respond  (what  genre,  or  strategy,  to  use).  Hence,  a  major 
variable  In  the  study  is  topic/year,  or  the  unique  topic 
offered  In  each  assessment  year.  These  are  identified  in 
this  report  as  follows: 


The  complete  prompts  are  included  In  the  Appendix  as  Table 
1,  and  are  typical  of  writing  assignments  offered  in 
schools,  particularly  for  the  purposes  of  making  over-all 
Impressionistic  quality  ratings  of  the  kind  used  in  the 
Drake  writing  assessment  program. 


1973: 
1974: 
1975 : 
1976: 
1977: 
1978: 


The  [EVENTl  topic 

The  [BEl  a  [NDN-HUMANl  thing  topic 

The  IBEl  ANOTHER  tPERSONl  topic 

The  tlNVENTlONl  topic 

The  [SPACE  OBJECT]  topic 

The  flNFLUENCEl  topic 
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Two  impressionistic    quality    '•^V"^'.  '''i  L  Jhl  ^tHv 
essay:     an  orlqinal   rating  (Old  Score)  ass.  gned  to  the  essay 

by  teachers  at  Drake  the  year 

which  the  essay  Is  compared  to  all  others  written  on  the 
:S:e  toMc.  including  essays  by  students  at  all  S^^de  lev- 
els, I.e.,  by  the  full  schcol  population  at  the 
?elung.  i  seLnd  quality  rating  (New  Score)  -as  assigned 
to  ealh  essay  in  the  sa.ple  during  a  P^^I^^^jr^^^;;^  ?J 
the  essays.  In  which  the  essay  is  compared  to  all  others  in 
the  sample  (see  Table  2  in  the  Jfppendlx). 

TK-    Student     writers.      The    populatior.    of  students 
attenJ^Sg  'If  J^^raS'c^'s^'grake  Hlsh%Chool .  J"  H-'j/^-^V; 
California,  remained    reasonably    stable    J"*^  "9.  ^J*  ^J*f" 
encompassed    by    the  study  and  consisted  of  middle  to  upper 
middle  class    students,    with    few    students    -ho    could  be 

cia^l^t^Hxed  as  educationally  ^-<1  •^•PPJ^intr^li'ton^ 
dialect,  or  Engl  I sh-as-a-second-language  students,  ^^^^lon 
a  Iv  handicapped  or  learning  disabled  students  of 
S  slexia  or  other  perceptual  disabilities)  were  not  identi- 
?  ed  aS!d  Isolated  or>e™,oved  from  tN^  population  SBmpUj, 
either  before  or  after  testing.  On  the  average,  a  consider 
able  percentage  of  Drake  students  go  on  to  two-year  or 
four-year  colleges. 

IhC  ScllQQl  ULltina  ECQflLani.    During  the    years  f 
i  assessment     program,     from  1967  through  the  present,  English 

'  ?lachers  at  Drake  must  be  described    as    exceptional  y  able 

^eacherr  of    writing.      The    core     staff  In  the  early  years 
au?ho?ed  a  series  of  composition  teaching    guides    for  Ginn 
(Liv°n!    It    b\V  1965)  Ehich  are  still  widely  Popular.  The 
lame  g^oup^as  active  in  leadership  roles  in  the    C.l  fornia 
issoclatlon    of  Teachers  of  English  and  provided  the  ^^S"* 
n^m?er  of  participants  from  one  school  in  the    "tab  Ishment 
anS    initial     sessions  of  the  Bay  Area  Writing  ""rljej.t.  Iht 
assessment  program  at  Drake  was  Introduced    by    Kate    »> 'Jk 
h^^n  Th^n'  Sep^tment  chair,  based  on  the  model  of  so-ca  led 
holistic  scorino  used  by  the    Educational     Testing  service, 
SnJ    fItJ"U«-.".t.d\hrou9l.  the  Bay  .rea  H''t ""9  Project 
to  local  school  Sites  throughout  the  country.    «1though  the 
Sr,ir  {..ch?^,    practices  Sey  not  yet  "*  JVPU.I  of  -rlting 
.^«><-hinn  In  the  country  as  a  whole     (Applebee.    19801.  tney 
^epreseSJ    ^    CanguJrd  approach  tc  writing  InstructJon. 
^Mch  ^eJps  ?o  assure  ?l,e  authbr  that  insights  into  student 
^riUng    development    gained     in    the  present  study  --ll  not 
au  cklv  become  irrelevant  as  improved  instruction  In       • t i hg 
Thlngel    tSe  naiure  and  course  of  writing  development  In  our 
culture. 

Ibfi  UftlCSiiQDlltlC  fl"U^i'J?H2fn?^  a2ami^'ra?"g!" 
The      general     procedures    for    obtaining     quality  Ratings, 

^  hereafter  called  holistic  scores  following  usage  in  the  pro 

f  fession,    were    similar    for  both  old  scores  and  new  scores, 

and  are  described  In  detail  elsewhere  by  Keech    l^Bla).  In 

summary,    the    procedure  Involves  rapid-impression  rating  of 
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student  essays  by  tv^^o  independent  raterst  each  using  the 
same  range  of  scoring  points  (in  this  instancet  a  scale  of  1 
to  9  with  1  awarded  to  the  poorest  papers  and  9  awarded  to 
the  best  papers  in  the  sample)*  Inter-rater  reliability  is 
improved  by  pre-tralning  using  a  scoring  guide  and  saitple 
paperSf  or  *anchors»*  which  illustrate  typical  qualities  of 
writing  to *be  found  at  each  point  In  the  scoring  range* 

Readers  in  the  pooled  re^scoring  of  the  sample  for  this 
study  included  teachers  who  had  read  in  the  original  Drake 
assessment*  as  well  as  additional  BAWP  teacher  consultants 
experienced  with  the  Drake  approach  to  holistic  scoring. 
The  holistic  scores  thus  obtalnedp  both  ojci  and  newt  are 
representative  of  conventional  and  traditional  impressionis- 
tic quality  ratings  of  student  work  which  have  been  treated 
as  dependent  variables  in  numerous  evaluations  of  writing 
Instruction*  and  have  themselves  been  subjected  to  repeated 
analysis  by  researchers  attempting  to  identify  the  factors 
which  influence  raters'  judgments  during  these  and  similar 
scorings.  (Freedmar»  1981;  Freedman  and  Noldt  1977;  Has- 
wellt  1980;  Diederlchf  197*). 

5cl£ctiQD       L^itSi  fcr  Analysis 

The  .three  longitudinal  cohorts  provided  a  total  of  90 
potent iaf^^ase  studies  of  students  performing  on  writing 
tasks  over  four  .years*  Since  exhaustive  analysis  and  coding 
of  all  90  cases  was  not  possiblet  ten  to  twelve  cases  were 
selected  for-n  each  cohort  (or  one  third  of  the  total  sa-nple) 
for  case-study  treatment  which  will  provide  the  bulk  of  the 
findings  reported  In  this  study. 

The  primary  criterion  for  selection  of  cases  of 
interest  was  the  appearance  of  a  non-chronological  sequence 
of  performance  quality  ratlngst  as  indicated  by  an  Adjusted 
Ranking.  This  ranking  combines  the  rank-ordering  of  the 
original  holistic  sccre  (DS»  in  which  the  paper  was  compared 
to  all  other  papers  on  the  given  proirpt)f  the  new  holistic 
score  (NSt  in  which  the  paper  was  compared  by  the  same 
raters  to  papers  on  all  prompts) »  and  a  forced-choice  rank- 
ing (FCt  In  which  2  Independent  raters  compared  the  papers 
of  one  student  only  to  other  papers  by  that  same  student. 
The  combined  rankings  allow  an  adjusted  ranking  (AR)  which 
reflects  all  3  fields  of  comparison  andt  thusi  all  of  the 
criteria  likely  to  be  used  by  teachers  or  raters  in  evaluat- 
ing student  samples  to  determine  irnprovement  in  writing 
ability.  Ideallyt  the  AR  was  designed  to  remove  misleading 
effects  of  different  comparison  fieldst  and  to  produce  a 
ranking  which  agreed  with  the  chronological  order  in  which 
the  papers  were  procuced .  The  case  whose  rank  in gs  were  not 
••correctable*  by  the  AR  were  those  chosen  for  study;  i.e. 
student  papers  whose  AR  did  not  conform  to  the  expected 
chronological  direction  of  change  (one  to  f our t  lowest  to 
highest  AR »  freshman  to  senior)  were  chosen  as  case  studies. 
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□riiiinallyt  we  had  also  Intended  to  select  case  studies 
according  to  a  second  criteria,  which  Involved  comparison  of 
a  student's  four  new  scores  with  a  mean  NS  in  that  student  s 
cohort  during  the  four  topic/years.  It  seemed  clear  that 
topic  ef^fect  was  reflected  In  mean  new  scores  for  each 
cohort  (e.g.  the  mean  US  for  Cjhort  I  rises  dramatically 
between  freshman  and  sophomore  yers— +2.%^  points--by  far 
the  larjest  gain  for  any  cohort  In  any  year,  see  Table  3}. 
However,  this  phenoirenon  may  also  reflect  an  age  effect,  or 
cohort  effect,  with  other  grade  levels  Improving  less 
between  the  two  years.  In  any  event,  we  Intended  to  select 
as  anomalous  cases  those  students  whose  new  ^scores  did  not 
conforn  to  the  expected  ranking  of  scores  based  on  the  mean 
NS  ranks  of  his/her  cohort  (.*hcther  or  not  these  progressed 
from  the  expected  chronological  sequence  of  development). 
He  realized,  however,  that  the  very  pattern  of  NS  mean  ranks 
for  each  cohort  may  reflect  the  phenomenon  of  interest;  I.e. 
topic  effect  Influencing  task  construction,  and  thereby 
Influencing  holistic  score  as  a  measure  of  performance. 
Thus,  we  selected  case  studies  solely  on  the  basis  of  AR 
anomalies;  i.e.  AR's  which  did  not  progress  in  a  direct  J «ne 
from  lowest  to  highest  over  the  span  of  four  topic/years. 

As  a  final  criterion  for  selection  of  case  studies,  we 
eliminated  these  cases  which  seemed  problematical  ,  or  of 
less  interest  for  a  variety  of  reasons:  e.g.  poems,  lowest 
level  writers  whose  papers  evidenced  scribal  difficulties  or 
too  little  writing  to  task  analyze,  highest  level  writers 
whose  score  ranks  were  skewed  due  to  the  "ceiling  effect*  of 
consistently  high  level  performance  (though  some  of  the 
latter  were  Included  in  our  sample,  if  it  was  possible  to 
make  clear  forced  choice  rank  decisions  about  the  papers). 


Initially,  we  tried  to    adapt    existing  classification 
schemes  to  the  papers  In  our  sample,  and  roughly  categorized 
them  In  terms  cf  "narratives"  (stories),  "descriptive  writ- 
ing"   and    "exposition"  (commentary,  argument).     However,  we 
found  that  this  rough  classification  did  not  do    justice  to 
certain    distinctions  In  the  writing  sample  that  were  criti- 
cal for  description  of  task  construction  In  all   Its  complex- 
ity.     We  realized  (and  other  text  design  research  suggests) 
^hat  students  generally  seem  to  have  some  notion  of  literary 
"genres"    on    which     they  model  their  writing,  but  that  they 
also  often  combine  elements  of  traditional  genre  categories 
to    produce  a  text.     For  example,  we  noted  that  some  writers 
chose  to  use  a  narrative  "schema"  (telling  a  story)  in  order 
to    dramatize     an     Idea.    Other  writers  were  able  to  express 
the  same  or  slirflar   Idea,    equally     successfully,     by  using 
expository    prose     (I.e.  a  hierarchical  arrangement  of  argu- 
ments in  an  explicit  discussion).     Sinllarly,     some  writers 
chose    to  reveal  a  character  "they  would  liVe  to  be"  by  nar- 
rating a  key  event  In  that  character's  life.      Others  chose 
to    describe  the  character  in  terms  of  its  habitual  actions. 


In  order  to  capture  these  d i st ? net i ons >  we  found  it  both 
necessary  and  useful  to  separate  task  construction  into  two 
elementSff  expressed  in  our  typology  as  •^r  hetor  !  c«  1  function" 
and  "discourse  stratejy."  Essentiallyt  we  made  a  distinction 
between  the  form  and  the  function  of  a  text. 

By  rhetorical  function»  we  mean  »What  does  the  i£Xl 
do?"  "How  does  it  function  for  the  reader?"  Function  thus 
involves  both  the  topic  focus  and  the  rhetorical  effect  of  a 
given  text  m  By  discourse  strategyt  we  mean  "How  is  the  text 
structured?  What  arc  the  fflfiaas  through  which  a  rhetorical 
function  is  achieved?  What  is  the  form  of  the  text?" 
Althoughf  as  we  shall  sect  certain  strategies  are  conven- 
tionally associated  with  particular  functions  (giving  rise 
to  more  tr  ad  1 1  ional  text  classification  schemes ) t  wr I ter  s 
can  and  do  successfully  employ  a  variety  of  different  stra- 
tegies to  fulfill  particular  functions.  See  Figure  2  and 
the  discussion  following. 
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There  are  three     analytlcany     distinct    functions,  or 
"things  that  a  text  can  do"  (though,  see  discussion  below  of 
phatic  and  performative  text  functions  not  included     in  our 
tvpology):     a     text     may  (I)  -tell  a  story,"  (ID  -reveal  an 
entity,-  and/or  (III)  -express  an  Idea.-  Each  of  these  func- 
tions    Is    associated  with  a  particular  topic  focus  and  rhe- 
torical effect,  as  mentioned  above.     In  Function  I,  -telling 
a     story,-    the  topic  focus  is  on  a  single  event  or  sequence 
of  events,  and  the  rhetor  leal  effect  Is    to    relate  -change 
over  tine.-  In  Function  II,  -revealing  an  entity,-  the  topic 
focus  is  on  a  single,  concrete    entity     (object,     place)  or 
Identity,     (character,  animal  or  human,  real  or  fantasy)  and 
the  rhetorical  effect  Is  to  relate  -that  which     remains  the 
same    over  time.-  In  Function  III,  -expressing  an  idea,-  the 
topic  focus  Is  on  an  Idea,  an  abstraction,  or  a  classifica- 
tion   scheme,     and  the  rhetorical  effect  Is  to  relate  rela- 
tionships between  entitles,  events,  writer,     reader,  and/or 
ideas    existlnq    beyond    time.-    Each    of  these  functions  is 
achieved  through  the  use  of  discourse  strategies. 

Strategies  provide  structure  for  the  discourse  unit 
known  as  the  -text.-  As  there  are  three  analytically  dis- 
tinct functions,  so  there  are  three  strategy  types:  (A)  nar- 
ration, (B)  description,  (C)  evaluation  (commentary).  While 
strategies,  as  a  whole,  provide  structure  for  individual 
texts,  the  strategies  themselves  contain  an  Internal  struc- 
ture of  elements,  termed  -propositions,-  concerning  events, 
descriptions  and  co^entarles.  Strategies  are  defined  by 
the  organization  of  the  propositions  they  contain.  Indivl 
dual  propositions  In  a  given  text  may  be  Identified  in  terms 
of  foraal  characteristics.  EyfiQl  propositions  have  the 
form:  ^x  happened.?  QfiSCLiBti^fi  propositions  have  the  form, 
"x  is,  was,  or  happens.-  CQOHDfiDlatX  propositions  have  tne 
form:  -y  believes,  thlnics,  feels  that  x  happens,  happened, 
is,  or  was.-  Furtherirore,  commentary  propositions  may  take 
the  form:  -z  believes,  thinks,  feels  that  y  believes, 
thinks,  feels  that  x  happenis,  happened.  Is  or  was.-  That  is, 
commentary  propositions  may  be  recursive. 

ECQBQSUicn  SLtUClUtC  Although  the  classification  of 
propositions  Is  sometimes  text  dependent  (e.g.  -John  Is  my 
friend-  could  be  either  description  or  comment  depending,  on 
whether  -John-  or  -friendship-  Is  the  topic  focus  of  the 
text),  there  do  seem  to  be  certain  characteristic  markers  of 
each  proposition  type:  Event  propositions  are  marked  by  use 
of  the  Bait  tfinSft  (except  for  copulas).  Descriptive  propo- 
sitions are  marked  by  use  of  the  BLCfifiDt  IfiDSC  (and/or  CS&£ 
tftQ5C  capulas).  Comirentary  propositions  are  marked  by  the 
following: 
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!•      Explicit     statement    of    opiniont    wisht     desiret  etc« 
(subjunctive     mood?)     such    as  wish,**        think,''  I 

hope." 

2.      Negation,  as  In  •He  didn't  go.*' 

3m      Modals,  conditionals  (should,     would,     could,    can,  be 
abl e)  . 

4*      Comparat  ives ,  superlatives. 

5.      Abstract  subjects  (marked  by  indefinite  article  or  mass 
noun) • 

6*      CI f chesf  aphor I sms« 

It  is  important  to  point  out  here  that  each  strategy- 
narration,  description,  and  evaluat  icn***conta  ins  alt  three 
types  of  propositions.  It  Is  not  the  presence  or  absence 
(or  relative  amount)  of  proposition  types  that  determines 
overall  strategy  but  rather  the  pattern  of  organization  of 
the  propositions,  Wihich  structures  the  text,  and  therefore 
defines  strategy.  For  example,  strategy  A  (narration)  Is 
composed  of  a  chronological  linking  of  "events*  which  form 
the  spine  of  the  text  and  through  which  the  text  is 
developed.  Events  thus  serve  as  the  "core  propositions*  of 
narrative  strategy.  In  addition,  a  narration  may  contain  a 
great  many  incidental  or  supportive  commentary  and  descrip- 
tive  propositions,  but  these  are  linked  to  individual  core 
propositions  and  not  to  one  another.  Similarly,  in  strategy 
B  (description),  the  core  propositions  are  descriptive;  com- 
mentary  and  event  propositions  provide  support  for  Indivi- 
dual descriptive  propositions*  In  strategy  C  (evaluation), 
the  core  propositions  are  commentaries:  event  and  descrip- 
tive propositions  are  supportive.     See  Figure  3  following. 
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It  is  also  luportant  to  note  that  the  organization  (Unking) 
of  core  propositions  within  each  strategy  may  take  several 
different  forms.  For  examplet  the  event  propositions  within 
a  narrative  strategy  may  be  linked  to  an  overarching  event 
("the  story")  and  to  one  another  as  elements  of  the  "plot- 
ted" narration;  I.e.  protagonlstt  goalt  compl  Icat  iont  climax 
and  resolution.  Or t  event  propositions  may  be  linked  only 
to  one  another f  as  discrete  event  units  through  which  a  main 
character  proceeds.  Each  of  these  event  units  may  contain 
plotted  story  elements;  e.g.  In  a  "picaresque"  narratlont  or 
these  discrete  event  units  may  be  linked  to  one  another 
without  containing  plot  elementst  but  rather  to  serve  as 
point  of  refsrcncet  stages*  or  periods  in  the  "llfe'^  of  a 
main  character  <such  as  epict  historfcal»  or  certain  forms 
of  biographical  narration). 

Simllarlyt  descriptive  propositions  may  be  ordered 
chronologically  to  produce  a  sort  of  "recipe"  (a  temporally 
ordered  set  of  procedures) t  or  they  way  be  ordered  spa- 
tiallyt  assoc fat lonally,  or  focally.  Commentary  proposi- 
tions may  be  linked  hierarchically  (In  what  we  tend  to  think 
of  as  "formal"  or  structured  argument) t  or  they  may  be 
linked  locally  or  a ssoc lat iona  1  ly  (see  Pringle  and  Freedman* 
1981). 

EclDaU  Ixcea*  As  mentioned  abovct  there  are  conven- 
tionalized (standard)  associations  o^  particular  strategies 
with  particular  functionst  which  we  have  termed  "primary 
types":  Function  I  (tell  a  story)  with  strategy  A  (narra- 
tion); Function  II  (reveal  an  entity)  with  strategy  3 
(description);  Function  III  (express  an  idea)  with  strategy 
C  (commentary),  the  "primary  type"  matching  of  function  and 
strategy  may  in  fact  be  expressed  In  several  different  wayst 
depending  on  the  sub-type  of  strategy  used  (I.e.  depending 
on  the  way  In  which  core  propositions  are  linked).  For 
example,  a  writer  may  tell  a  story  using  several  different 
types  of  narrative  strategy*  or  may  reveal  an  entity  using 
various  styles  of  description.  The  following  discussion  of 
sample  papers  will  reveal  both  the  basic  association  of 
strategy  and  function  In  these  "primary  types,"  and  the 
range  of  possibilities  within  each  strategy  classification. 

Primary  type  I A  (in  which  a  story  is  told  through  nar- 
ration) may  be  realized  through  what  we  have  termed  the 
"picaresque"  or  the  "plotted"  strategy.  Use  of  a  plotted 
narrative  strategy  to  tell  a  story  way  be  clearly  seen  in 
case  #110/73.  Picaresque  narration  is  used  In  case  #107/75» 
on  the  other  hand,  to  serve  the  same  function  of  telling  a 
story. 

Primary  type  IIB  (In  which  description  Is  used  to 
reveal  an  entity)  is  Illustrated  in  case  #420/78  (spatial 
description),  330/76  (chronological  description),  220/7* 
(focal  description),  and  326/75  and  108/75  in  which 
different    forms    of    assoc iat ional     description    are  used 


(i3?6/7*i  constituting  a  "self -declarator /*  monolog  through 
which  th«  entity  is  revealed,  and  #108/75  constituting  a 
"slice  of  life*  depiction  of  a  character). 

Primary  type  IIIC  (in  which  an  Idea  is  expressed 
through  evaluation  and/or  commentary)  may  similarly  be  real- 
ized in  a  variety  of  ways.  Perhaps  the  clearest  (or  most 
fafrillar?)  examples  of  such  •expository"  writing  are  those 
in  which  the  core  propositions  (comments  and  evaluations) 
are  linked  hierarchically  In  the  form  of  a  logically  ordered 
argument  as  in  case  #215/76.  It  Is  also  possible  to  evalu- 
ate an  Ideat  or  coirirent  upon  a  relationship  between  things, 
people  and/or  ideas,  by  organizing  coirmentary  propositions 
In  an  assoc iat  lonal  cr  focal  way.  Often,  the  combination  of 
assoclatlonal  or  focal  C  strategies  with  III  purpose  takes 
the  form  of  "musings"  or  "reveries"  about  a  relatively 
abstract  topic  as  In  cases  #230/77  and  #127/73.  ^ 

In  addition  to  these  primary  type  combinations  of  fJnc- 
tion  and  strategy,  there  are  a  variety  of  ways  In  which  dif- 
ferent strategies  may  be  successfully  used  to  fulfill  the 
same  function.  As  Indicated  above.  It  Is  equally  possible 
to  reveal  an  entity  (a  character)  though  a  dramatization  of 
that  character's  life  (or  a  koy  episode  In  that  character's 
life)  as  it  Is  to  straightforwardly  describe  that  character. 
Similarly,  one  may  express  an  Idea  by  "showing"  that  Idea 
(through  narration)  or  by  "tel 1 i ng"  that  Idea  (through  com- 
mentary, cf.  Kaplan  and  Keech,  1980). 

However,  one  combination  of  strategy  and  function  that 
is    not    theoretically    possible.  Is  the  use  of  any  strategy 
other  than  narration  In  order  to  tell     a    story.      That  Is, 
Function  I  can  only  be  fulfilled  through  strategy  A,  since  a 
"story"  necessarily   Involves  depiction    of    a  chronological 
sequence    of    events    mirroring  change  over  time  in  the  real 
world.    This  seems  to  be  related  to  a  wore    general  pattern 
of    uni-dlrect  lonal     constraints  on  possible  combinations  of 
function  and  strategy,    that  Is,  there  Is  a    progression  In 
level     of  abstractness  from  Function  I  to  Function  III;  from 
the    exper ientlally-based     "revealing    an    entity,"    to  the 
abstractness     (and     lack    of  temporal  reference)  involved  in 
"expressing  an  idea."  Similarly,  there  is  a  progression  from 
the    preponderantly  exper ientially  based  language  of  A  stra- 
tegy (narration)  through  the  concrete  language  of  B  strategy 
(description)     to     the  abstract  language  of  C  strategy  (com- 
mentary, evaluation).    Me  noted  that  the  common    pattern  of 
combining    these    aspects     Is  that  relatively  concrete  stra- 
tegies may  be  used  to  fulfill  more  abstract    functions,  but 
that    the    reverse  does  not  tend  to  occur.     That  is,  A  stra- 
tegy may  be  used  to  fulfill  Functions  I,  II,  and  III;  but  C 
strategy    may  not  be  used  to  fulfill  any  function  other  than 
III  (at  an  equally  "abstract"  level).     Similarly,  Function  I 
can    only    be     fulfilled    through  use  of  the  exper ientially- 
based  language  of  strategy  A,     However,  Function  11     can  be 
fulfilled    through  use  of  A  or  B  strategy.    This  directional 
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pattern  of  strategy/function  combination  can  best  be  under- 
stood with  reference  to  the  follo-ing  chart- 

I  A       Generalized  Experience 


II 
III 


B  Concrete  -facts*  from  Experience 
C        Abstraction  from  Experience 


Several     examples    of    strategy/function  combinations 
seemed    particularly  salient  to  us.  s.nce  -e  have/«c  3 
them  In  literary  genres  «  -ell  as  J^^^^J^^y  narra- 

under     analysis.  ^°f  "^!;°\oJm    of    ep  c  narrations 

H;= 

ILltcllTU  "  C.S..  «10/73  and  .XIO/M. 

may  function  both  to  tell  a  *^°^y?j"hed  through  use  of  a 
(Function  I-II).  This  may  a"°™Pl»*;jJ  J^J^^^j^HaMy. 
single  strategy  U,  for  ex.mple    n  ^^^se ,m/73)  • 

Functions  1  and  III  '^V;^*?""Jcept  or  an  Idea,  through 
serve  to  express  an  ^^^^  "J^^^J^^Pjion.  as  in  case  lOB/7%). 
narrative    strategy   n  !^ki„.  u,ith  dual  strateg  es  in 

Finally,  dual    ^""^tions  «ay  combine  -I  thdu^^ 

transition,     back     and  /^'^^^ »  .^.^e  r"  core  proposi- 

the  body  of  a  text).  '"^  '"J^S-^tJ  °3  to  one  another. that 
tions  of  two  strate3ies  J^jJ^^^^Jhe  simultaneous  develop- 
the  text  Is  structured  In  terms  ^'^^p*^^  J^'^^  evaluation), 
ment  of  two  stratejles.  ^l^-  narrative  a  strategy 
Particularly  clear  e**"?' «  -auJob  ographical  essay- 

combinations    can     be  .ten?  the  development  of 

^^"^?^.r  !«:cTfuil?rof  eu'-aSlreness-)  over  ?ime  Cl- 
an Idea  _ase  302/78).  Similarly,  descrip- 
III/AC  combination,  see  *=f*^,;";'^  „ted  as  strategies  to 
tion  and  corrmentary  are  '"^"f idea  while 
fulfill   a  mixed  IIMI    ^"jctlon  ««JP^«»  JfJ^,  Combination 

^-.riron^f 

T.?l\Vof  Tg  ^arSrJ-irab-sJJart  Ta^  age  In  an  otherwise 
descriptive  text  serving  to  reveal  an  entity. 
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administration  of  a  reliability  check  afiong  three  raters  to 
ensure  the  repl  i  cab  II  i  ty  and  consistency  of  the  coding  pro* 
cess.  Raters  v#ere  tralnedt  as  for  holistic  scoring  ses- 
sionSf  through  the  selectiont  codfngt  and  discussion  of 
anchor  papers  chosen  as  representative  "types."  Gnce  rater 
agreement  was  achieved  in  these  preliminary  discussionst  a 
sub^sample  of  2*  papers  was  chosen  randomly  from  among  the 
case-^study  papers:  *  paperst  one  from  each  topic/year  of 
each  of  the  six  cohorts.  Three  raters  coded  papers  indepen- 
dentlyt  In  terms  of  function  and  strategyt  assigning  codes 
from  1  to  6  (which  encompassed  dual  functions  and  dual  stra- 
tegies) to  each  papert  one  number  code  for  function  and  one 
for  strategy. 

He  realizedf  during  discussion  of  the  anchor  papers 
prior,  to  the  coding  sessiont  that  raters  seemed  to  agree 
fairly  closely  about  th€  most  "salient*  or  "primary" 
^unctlonCs)  of  a  given  textt  although  raters  occasionally 
perceived  different  secondary  or  incidental  functions  within 
the  same  text.  Slmllarlyt  raters,  were  able  to  agree  on 
salient  or  primary  strategy  more  easily  than  on  secondary  or 
incidental  strategies  which  they  perceived  In  a  given  text. 
Thereforet  we  decided  to  compare  the  independent  rater  cod- 
ings in  the  reliability  check  In  terms  of  a  minimal  cri- 
terion of  agreement:  I.e.t  if  raters  agreed  on  at  least  one 
element  of  strategy  and  one  element  of  function  within  a 
given  textt  9it  considered  that  to  constitute  rater  agree- 
ment. For  example  f  if  rater  #1  assigned  a  function  code  of 
III  to  a  textt  while  rater  #2  assigned  a  code  of  II-III  to 
the  same  textt  they  were  considered  tc  be  in  agreement  con* 
cernlng  the  primary  cr  most  salient  function  of  that  text. 
If  two  or  more  raters  failed  to  agree  on  all  aspects  of 
function  and/or  strategyt  we  decided  that  a  rater 
discrepancy  existedt  and  looked  to  see  if  this  discrepancy 
could  be  resolved  through  a  different  rater  comparison.  For 
examplet  ff  rater  il  assigned  a  function  code  of  lit  and 
rater  #2  assigned  a  code  of  Illt  this  constituted  a  rater 
discrepancy;  there  was  no  aspect  of  function  on  which  they 
could  agree.  Out  of  the  24  papers  in  our  reliability  check 
samplet  there  were  7  cases  In  which  such  rater  discrepancy 
occurred.  In  every  cne  of  these  casest  however t  discrepan- 
cies occurred  only  between  2  of  the  raters.  Since  there 
were  3  Independent  raters  part  lei  pat  ingt  thi  s  al so  meant 
that  at  least  2  out  of  3  raters  agreed  in  every  single  case, 
thust  discrepancies  between  2  raters  occurred  in  30  percent 
of  the  cases;  discrepancies  between  all  3  raters  never 
occurred  (I.e.  there  were  no  cases  i,n  which  no  agreement 
could  be  reached  on  both  function  and  strategy);  and  agree- 
ment between  at  least  2  out  of  3  raters  occurred  in  100  per* 
cent  of  the  cases. 


Given  our  experience  with  the  reliability  check  (and 
our  feeling  that  the  typology  w35--and  still  is--subject  to 
some  re  ^  Ine-nen  t )  t  we  lecided  to  code  our  larger  sample  of 
papers  according  to  the  following  procedure: 
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Each  paper  w»s  coded  Independently    by    2    raters,  who 
assigned  independent  codes  to  function  and  strategy. 

Agreement  between  the  two  raters  was  assessed  according 
to    the  sa«ie  criteria  as  In  the  reliab.lity  check,  i.e. 
identical  coding  of  one  or    more    aspects    of  *=u"ctlon 
and/or    strategy    constituted    agreement  on  the  primary 
characteristics    of    that    function    and/or  strategy. 
Furthernore,     if    raters     agreed  on  one  aspect  of,  say, 
function,  but  disagreed  on  one  or  more    other  aspects, 
we  assigned  the  agreed  upon  code  to  a  -primary  ^""^t'O" 
category*  on  our  final  coding  sheets,  and  assigned  the 
Siscfepant      rater      codes     to    a    -secondary  function 
category.-  For  example:     rater  #1  reports  a  code 
III,  rater  #2  reports  a  code  of  I.  the  primary  function 
code  would  be  I.  and  the  secondary  funct  on  code  would 
be     III;  or,  rater  #1  reports  a  cede  of  I-III,  rater 
reports  a  code  of  I-II,  the  primary  ^""^1^?"  ! 
and    the     secondary    function     code  is  ^J'^ 
way,  we  attempte^l  to  preserve  a     '^t  i  net  ion  between  the 
most    strongly    perceived,  or  most  salient  function  (or 
s??ategy)  of  Ihe  text,  and  those  which  seemed  Inciden- 
tal, ambiguous,  less  clearly  defined,  or  which  may  have 
leen  the  product  of  rater  -error.-  (Mote:  /he  results 
reported   in  the  following  section  are  based  exclusively 
on  prliiary  function  and  primary  strategy  codes,  for  two 
reasons:     so     that    the    effect    cf,  rater  error  may  be 
m"i°ired      (through      the^     exclusion      of  je""^ary 
categories,       to    which     discrepant     rater  "^^f 
assigned),  and   In  order  to  maximize  the  poss ib 1 1  I ty  of 
discovering  distributional  patterns  in  the  data. 

Cases  of  discrepancy  between  the  first  two  raters  were 
assigned  to  a  third  Independent  rater  fo*"  "^'"3-  *J 
m  the  reliability  check,  the  use  of  a  ^^^I'^^^'iHV^ 
resolved  most  discrepancies  m  the  coding  process. 
Including  the  cases  used  in  our  reliability  check,  our 
sample  totaled  252  papers.  Out  of  that  sample,  again 
including  reliability  check  papers,  60  discrepancies 
occurred  In  function  and/or  strategy  coding.  ( If  a 
single  paper  contained  discrepant  codes  for  function 
strategy.  It  was  only  counted  as  one  discrepant 
case.)  Of  the  60  discrepancies,  *9  were  clearly 
"solved  by  the  third  rSter,  and  U  were  ultimately 
coded  as  -unci assif iable-  (see  14  below). 

Finally,  where  two  raters  agreed  that  P^P^''^  wf^« 
unclassif  lable  ;iccording  to  our  typology  (see  discus- 
sion below),  these  papers  were  coded  as  -9-  (unclassif- 
lable).  Similarly,  If  -11  three  raters  failed  to  reach 
agreement,  or  If  the  third  reader  agreed  w  th  ,one  °f 
the  original  readers  that  a  paper  was  unci assi f i abl e , 
or  If  the  third  reader  alone  felt  the  paper  was  unclas- 
sifiable  (and  that  this,  therefore,  was  the  source  of 
original   rater  discrepancies),  the  paper  was  coded  as  a 
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•9.''  Since  function  and  strat^^gy  were  coded  indeoen- 
dently  of  one  another  t  It  should  be  noted  that  a  *9'» 
function  code  did  not  necessarily  entail  a  "9"  strategy 
code  (again,  see  belo><  for  discussion  of  these  cases). 

Th^*  most  frequent  source  of  discrepancy  In  coding 
appears  In  papers  which,  according  to  raterst  seemed  to  con- 
tain some  combination  of  description  and  evaluation  in  func- 
tion and/or  strategy  (22  out  of  60  discrepancies  in  the 
total  sample).  For  example,  there  were  papers  In  the  sample 
which  seemed  to  contain  a  great  deal  of  descriptive  language 
(B  strategy)  along  with  an  abstract  topic  focus  (generally 
indicating  a  III  function).  Raters  differed  In  coding  these 
papers  as  II/B,  III/C,  II/B,  or  even  II-III/B,  C,  or  BC. 
Where  a  discrepancy  existed,  the  third  rater  generally  pro- 
vided a  II-III  function  code,  which  resolved  the 
discrepancy.  When  one  rater  Indicated  a  II-III  function, 
where  the  other  Indicated  III  function,  where  was  no 
discrepancy  and  the  paper  appears  In  the  final  coding  as 
having  a  III  pri-nary  function  and  a  II  secondary  function 
(as  opposed  to  the  II-III  primary  function  provided  through 
third  reader  resolution).  Thus,  several  essentially  similar 
papers  appear  In  our  final  coding  as  either  II-III  or  III 
function  papers  (having  either  a  B  or  a  BC  primary  stra- 
tegy). Similarly,  several  papers  seemed  to  contain  rela- 
tively abstract,  evaluative  language  (C  strategy)  along  with 
a  concrete,  descriptive,  "character  fccused'*  function  (II). 
Again,  these  papers  appear  In  the  final  coding  generally  as 
II-III/SC  types.  However,  some  were  finally  coded  as  Il/C^s 
and  some  as  II/P's  with  III  and  C  as  secondary  function  and 
strategy . 

These  di screpanclest  and  the  general  ambiguity  in  the 
final  coding  of  such  papers,  seem  attributable  to  two  fac- 
tors: topic  effect,  and  what  we  have  termed  the  "bleeding 
strategy  effect"  on  rater  perceptions.  Topic  effect  Influ- 
enced the  coding  process  (as  distinct  from  the  distribution 
of  task  constructions,  discussed  below  In  "Results"),  to  the 
extent  that  some  writers  appeared  to  be  attempting  to  honor 
topic  constraints  "Independently"  of  the  primary  function 
and  strategy  they  established  In  a  given  task.  This  was 
especially  clear  In  responses  to  topic/years  197^  and  1975, 
where  students  were  asked  to  write-  about  a  "non-human  thing" 
(1974)  or  a  "person  other  than  oneself"  (1975)  that  they 
"would  like  to  be."  Students  appeared  to  vary  In  construct- 
ing the  task  so  that  either:  1)  most  emphasis  was  placed  on 
"describing"  the  character,  with  a  nod  to  topic  constraints 
consisting  of  some  evaluation  of  the  desirability  of  being 
that  character;  or  2)  emphasis  was  placed  on  either  describ- 
ing or  evaluating  the  wrlter^s  "feelings  about  being  that 
character,"  with  a  ncl  to  topic  constraints  In  the  form  of 
descriptive  statements  about  who  or  what  that  character  was. 
Depending  on  the  relative  emphasis  a  writer     placed     on  the 
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central     character     -    -  ':h:"Ir"'%hr  - ' ' 

that     character,     and    <1«P«"«^  °n     f  J or  con- 

emphasized  description  or  evaluation  of  ^ ' *  ^^^jes  to 

Crete  subject,  '"f^ers  seemed  to  vary  in  ass. gn^^ 

:'£-:,iy.-t  r.::;;;::;-,-:.  • 

some  ambiguities  re.rain  in  the  final  codes. 

A  r^iatsd  toDic  effect  Influence  on  coding  can  be  found 
related  topic  eTTc  X  ^  secondary  strategies. 

language    In     their    ''•"'Jj* .^J  ?  2ere    an    eagle  I 

'"'l/'/r.Vl  "n«t"h  g,;  tn  '  ^e  i^it^nt.-  This  2se  of 
would    build     a    nest    nign     in     ^  strategy  code  to 

o?^:i5i?s''r;'s?s?in5s^;s  ?s«'rp;"p: ,  ; .  «hers  in  -hUh 

ever,  in  the  report  o^*^**"^  J'^^ ,?  "^"^  diitinguished  from 
having  a  C  secondary  strategy  category    codes  «ere 

-pure*  Il/B  types,  since  only  primary  categorY 

anal yzed . 

The  -bleeding  strategy-    effect  rater  P^^"^;'":^ 

,and    therefore    l^.r'rlctTotl^ ,>Vtr"l\  fincti oim^itegy 

"»'-  "-  -r-  ^;£hur/"-rnrt!or 
^•^:;:t^^:in^^l^"'^nrot•  s   fmed  thro^ 

?S-?h\n;rr!?Uono;''"co;™n  st^^^ 
narrative.    5  i.ril  arl  y.  commentary  .descr.pt  ion  a 

?rdea??5^  cr„nc?"he;.  |d  £  ev^'  'Sr  "aV  ^^^^^^ 
seems  that  when  a  writer  J^^^f     abstract  nouns. 

;-r-to'"?:if "  n;t;£n;  hK  Js^  .i.ar^ 

lu^tlgr  are  'e:p.;;c,n;  sUarahle--the  fact  that  they  are 
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is  evidenced  in  several  sample  papers  which  raters  agreed 
should  be  coded  as  II/C  rather  than  II/B  or  III/C.  However» 
nany  of  these  papers  (which  evidenced  the  "bleeding  stra- 
tegy"  effect)  were  ultimately  coded  as  I1/BC»  III/BCt  or 
Il-III/Sf  Cf  or  BC,  Againt  essentially  similar  papers 
^similar  ?n  terms  of  task  construction)  were  finally  coded 
in  a  variety  of  ways»  though  it  is  important  to  point  out 
that  the  essential  distinction  between  II/B  and  III/C  task 
constructions  was  maintained  In  the  f  iral  codes.  That  $s» 
none  of  these ^•amb I guous"  papers  was  ever  coded  as  a  I1I/C» 
although  the  III/C  ceding  category  does  contain  an  Internal 
range  of  variation  in  types  which  is  quite  distinct  from 
that  just  discussed. 

Essentially*  the  III/C  code  was  applied  only  to  papers 
which  expressed  an  idea  through  the  use  of  commeni^ry  cr 
evaluative  language.  However #  papers  In  this  general 
category  varied  considerably  In  terms  of  the  arrangement  of 
corr^mentary  propositions  they  contained.  Some  of  these 
papers  consisted  solely  of  assoc lat lonal  1  y  linked  "musings" 
about  a  particular  Idea  (e.g.  "the  danger  of  guns").  These 
might  be  termed  "stream  of  consciousness"  essays»  In  which 
the  author  appears  to  be  sustaining  one  end  of  an  open-ended 
conversation,  rather  than  presenting  a  clear  sequence  of 
logically  (hierarchically)  linked  arguments.  In  this  sense, 
they  constitute  what  Keech  has  termed  "engaged  dyadic  (open 
schema)  strategies"  (cf*.  "Toward  a  Developmental  Theory  of 
Discourse  Alms  and  Strategies,"  In  progress).  Othere  simi- 
lar,  papers  In  the  III/C  category  were  organized  focally 
around  a  specific  topic  or  Idea;  ^cr  example,  consisting 
almost  entirely  of  a  series  of  rhetorical  questions  concern- 
ing that  Ideat  addressed  to  a  fictlve  audience  Including  and 
extending  beyond  the  "test  reader.^  for  the  purposes  of  this 
analysis,  however.  It  seemed  most  Important  to  stress  the 
commonality  of  these  papers  In  coding  them?  they  all  seem  to 
represent  stages  in  the  development  towards  "classic"  expo- 
sitory proset  or  a  hierarchically  ordered  sequence  of  argu- 
ments through  which  the  author  anticipates  and  responds  to  a 
range  of  possible  Interlocutors  (the  readers  of  his  text). 
Furthermore,  It  Is  Important  to  point  out  that  the  use  of  an 
a ssoc la t ional 1 y  llnkel  strategy  does  not  preclude  the  pro- 
duction of  expert,  highly  proficient  and  successful  prose* 
There  are  degrees  of  proficiency  within  the  "stream  of  cons- 
ciousness" style ,  as  there  are  In  "classic  expos  1 1  ton  or 
argument,"  and  In  fact  within  each  type  of  task  construction 
we  have  presented  here.  Thust  the  grouping  of  different 
commentary  strategi  es  under  the  III/C  category  does  not 
obscure  differences  In  quality  levels  within  that  category 
any  more  than  does  the  grouping  of  variously  "good"  and 
"bad"  papers  under  any  other  category  of  task  construction. 

Aside  frorr  the  above  mentioned  d  I  st  i  net  I  onst  and  slmi- 
Tarltlesf  among  papers  that  may  have  been  obscured  In  the 
coding  process  (or  that  gave  rise  to  discrepancies  in 
coding),     there  was  another  relatively  large  group  of  papers 
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for  -hich  our  typology  could  ""J  »"2"'!'„\^Jm1''"'f  uniuln 
pap.r,  wer.  coded  ,s  ^""^'"ii'^'ticlls  ^f^bTe  ?or  on^'of 
or  strategy  or  both.  Papers  were  ""^'"f t^an  t«o 
two    reasons:    either  they  appeared  to  j^^^*^*  ^^^"xo^e 

functions  and/or  strategies    simultaneously     (and  therefore 

contained    no  clearly  defined  '°"  ^LJough  ^use 

or  thev  fulfilled  a  social  or  phatic  ^unction  ^^hrougn  use 
of  a  distinct  strategy  not  included  in  our  typology  which 
over^od^lni^Jetorical  function  included  in  our  typology. 

Examples  of  the  first  type  of  unclassifiable  paper 
includ^those  finally  coded  as  II/9's;  i.e..  those  with  a  II 
fJni?ton;  but  an  unclassifiable  strategy.  These  PJP«^^  -^^^ 
genera??;  written  in  response  to  topic/year  1977  (-choose  an 
object  to  take  into  space").  Typica.ly.  these  papers 
focused  on  the  decision-making  P--^""^^^^!^' J^^^  -^icj 
Object  to  take)  rather  thin  on  the  ^u.^Vlt '^^  of  the  object 
itself,  or  an  evaluation  of  that  object  as  a  -good  cjo'"' 
c  assif ieS  these  papers  as  having     II     functions  because 

cho  Us"'  5n  '.OS!  c««.  the  language  of  this  ^trat.gy  s 
na^r  tense,  and  appeared  to  be  "narration",  yet.  it  was  aiso 
Itrl  eCalua?ive  and  descriptive  of  the  objects  being  con- 
sidered^ TheU  papers  were  actually  quite  interesting  exam- 
D  es  of  what  ^ay  be  a  distinct  task  construction  type.  They 
pies  Of  T^J,^,,      to  be  simiKr  tc  the  autobiographical 

Tsrays:  ^drd  is' ?1 1^5/*?.    However,  where  the  -utobi  ogr  aph- 

«say  traces  the  dCi^elflBMDt  of  an  idea  o^^-"  t|T«  ^" 
minatlng     In     an  Increased  sel f -awareness  or  ""<^«^jt«"^  JS^' 
Jh"  11/9  paper  simply  mirrors  a  thought  P^-^cess-the  self  as 

^l^^o^^ij^^.  :?it%?ri?;°norrahiiJi^ 

lll?ultl6U  rstUer,  th^y  are  -P-^ing  the  process  of  .aki  g 
a  decision.     Narrative    past     tense     and     "^rrative  P^^^JJ 
tense     papers     are     alike:  -First  I  thought-then  »  t^^^^ht 
Jo^pared  with  "Hell.  I  could  take   .    But  it  needs  to  be 


Examples  of  the  latter  groups  of  unclassifiable  /^pers 
includ^    fetters.     speeches,     and  certain  -dialogues."  These 
yjn.rc  wpre  coded  as  "9's*  in  terms  o^    function    ami  stra 
feSy'    Ts  menUone".  the  function  of   such  papers  seems  pM- 
llVn,  tote  one  of  establishing  social  -PP°:^ 
or    solidarity.      They  .^^'^'"I'^tt    If    ^Vtt\l    or  no 

dents."    and     usually  '^^"^^^^IhJ  expression  Sf 

significance,    except    as    a    focus    for     the    expression  oi 
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concerrii  affectiont  Jnterestt  or  gratitude  on  the  part  of 
the  wrif^r  towards  another  person  or  persons.  For  this  rea- 
son, we  have  deliberately  excluded  these  papers  from  clas- 
sification accordinj  to  our  typology  of  "rhetorical 
discourse  functions  and  strategies."  Rathert  these  papers 
seem  to  prl'narily  fulfill  either  "phatic'  functions  (e.g. 
letters  to  loved  onest  friends  or  acquaintances;  speeches  to 
a  specified  audience  rather  than  the  •test-reader")  or  they 
fulfill  •performative*  functions  (e.g.  thank-you  letters* 
job  app 1 e cat  ions )*-al 1  of  which  are  oriented  more  towards 
establishment  of  a  social  relationship  between  writer  and 
ffctive  audiencet  than  towards  persuasively  communicating 
information  about  facts  of  the  world. 

&&^ull2  dad  tiyBcth£Sc:s 

As  a  first  step  in  this  research*  we  looked  for  corre-* 
1a<:ions  between  topic  and  score  in  terms  of  the  pivotal 
variable  of  "word  count."  We  discovered  thatt  as  had  been 
predicted!  word  count  correlates  significantly  with  new 
score  (NS)  for  our  sa^le  (n2  *  .10).  New  score  tends  to 
Increase  with  word  count.  Thereforct  we  hypothesized  that 
word  count  might  be  increasing  with  topic  (as  a  correlate  of 
task  construction  used  within  each  topic}i  thus  providing  a 
pivotal  link  ^between  topic  and  score.  Howcvert  we 
discovered  that  although  a  trend  exists  for  higher  word 
count  to  equal  higher  scoret  the  variance  of  word  count 
within  each  score  was  too  great  to  cite  word  count  as  a 
na jor  factor.  Furthermoret  word  count  did  not  seem  to  be 
affected  by  topics  in  consistent  ways;  e.g.  topic/year  1973 
generated  the  fewest  wordst  but  the  second  highest  mean 
scoret  while  topic/year  1976  generated  the  most  wordst  but 
the  fourth  highest  mean  score  our  of  six  topic/years). 

We  next  looked  for  correlations  between  topict  grade 
and  task  construction  as  a  preliminary  to  discovering  possi- 
ble relationships  between  topic  and  score.  Since  we  ulti- 
mately wish  to  examine  the  factor  of  task  construction  as  a 
variable  Influencing  quality  ratings  (or  change  In  an  indi- 
vidual student's  quality  ratings  over  time)i  we  felt  it  was 
first  necessary  to  dctemine  which  task  constructions  were 
favored  for  a  given  grade  within  a  given  topic/year.  then 
we  will  be  able  to  tell  through  further  analysis  whether  a 
student's  use  of  an  unexpected  (or  "unfavoredt"  "unpre- 
ferred")  task  construction  In  a  given  year  might  have 
affected  that  student's  score  (i.e.  was  that  student  trying 
to  do  something  new  or  different  both  from  what  he  had  done 
previouslyt  and  frc^  what  other  students  generally  did  in 
response  to  the  same  topic?  Did  this  Influence  his  score?) 
Future  analysiSt  thent  will  concentrate  on  case  studies  of 
students  who  deviated  from  the  "favored"  task  construction 
in  A  given  topic/yeart  as  compared  with  students  who  did 
produce  the  favored  type. 

The  results  as  reported  here  will  make  reference  to  the 
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variables  of  topic/year,  grade 

task  construction.  For  purposes  of  ""^^^^^^^J^^r  ables  of 
Construction  was  coced  In  terms  of  i"f  P^"^*",^ /f, ' ?o?. 
^'unction  and  strategy  (f/s).    These  arc  V.i  =  1/a; 

.owing     discussion^in    ter.s^o  _^^^^^^^  H'lH  BC*; 

1*1  \     I'h/ll     (p  K    va;ious  permutations  of  these  codes. 

'  u  i  »  -  Tll/AC;  5.3  '  Il-III/C;  etc.).  The  code  for 
such  as  3.*  -  Ill/*t.  5.3      *        *  ■9.»  as  discussed 

unclasstfiable    function    or     strategy     is  -f, 

above*  -  • 

X     I   /v.  =  r  107^      ♦•Event"  Tocic.     In  the  initial  exami- 

"3r"soirs:  J  nn^«^.d'ro"rd^rta:r5J°co..and  --rue 

had    actually     happened,    or  even  as   ii  ^-it.r  had  made 

the  narrator  spoke      It  -"'^^I^P^fd  TU'oT.TJsl'  using 

response. 

In  1973  overall.  «  percent  of  tj'/,"^?""'  "t"  oJher 
,/A    <1.1'  ':er««.":ere"    'tr  b:?^^  o«;  "Jine 

^^??:?rnt  '^rte  l^l^sf  nSnn}  -'-  co„talned  ev      ^  « 
cent  of  the  1973  student  "sponses.  These 

less    preferred  .^VP"    of    "rit  ng  are  of  P«»;"9„,„^^j  ^ 

rarr:uCe'•rt^n;/^^  ^7^:£T:  ';J;n^d'^i; 
jifdenf  ?r  rr^ir  r:rStnUg?«  '  ^^^l^i^^-^i^r^ 

To  senior  yea?,    l^ewer  freshmen       /VPe        «  Ji;,,', 
^a^-t  r.ua?^r-.S"edTn  .h  en^  f.  th^t-o  stra- 

^iraiii:^ei?Ls^:«::^^;-ngivpefe^  jiios^^rro-aii': 

this  topic,  seven  out  of  ten  170  percent  c^^^  solutions 
^aJd^^in  ro    %r  Lr^lO  Te^rn"  writing  a  3.3  ^ype  paper. 

o;°a  tota?  if  SuviS  «nlo;s  writing  1.1  type  papers  on  all 
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topics  combined^  7  of  them  (6^  percent)  were  written  in 
response  to  the  1973  (Event)  topic. 

The  197*  topic  produces  few^er  dominant  preferences. 
Only  among  freshmen  is  there  a  strong  central  preference? 
type  II/B  (2f2)  or  descriptive  strategy  used  to  reveal  an 
entity.  50  percent  of  the  freshman  papers  on  this  topic 
were  type  2 $2  while  of  the  Temaining  50  percent  of  the 
paperSf  32  percent  con ta ine d  some  II  function.  There  is 
also  a  weak  preference  among  sophomores  for  the  2f2  type  (30 
percent).  Juniors  and  seniors  in  this  toptc/yeart  howevert 
evidenced  no  clear  preference  for  this  type;  seniors  in  fact 
produced  as  many  3,3  papers  as  they  did  2f3'5  (both  at  20 
percentt  and  both  greater  than  the  percentage  of  2f2  paperst 
at  10  percent).  1 nterestlngly t  this  topic/year  was  the  one 
in  which  r^an  scores  within  each  cohort  were  highest;  i.e.t 
students  in  all  grades  writing  on  this  topict  on  the  aver-* 
age  8  tended  to  get  th'*ir  highest  scores  here . 

We  expectedt  inltiallyt  that  patterns  of  task  construe* 
tion  distribution  for  the  1975  topic  would  be  essentially 
Identical  to  those  In  197At  since  the  topic  constraints 
(expressed  in  the  promptt  or  "given  task")  are  so  similar 
for  each  topic/year.  However #  *1975  responses  show  a  clear 
pattern  of  preferencet  changing  over  time  (by  grade)>  unlike 
197^  responses.  In  fact  1975  responses  demonstrate  what  we 
hypothesized  as  the  expected  developmental  progressiont  from 
Itl  to  3t3  (freshman  to  senlort  exper  lent  i  all  yba  sed  writing 
to  abstract  exposition).  Freshmen  in  1975  show  a  weak 
preference  for  stories  (Iti:  27  percent)*  which  shifts  among 
sophomores  to  a  weak  preference  for  description  (2f2:  25 
percent).  The  amount  of  evaluative  responses  (3f3)  stays 
relatively  constant  from  freshmen  to  sophomores  (at  18  per- 
cent and  16  percent  respectively).  Among  junlorst  howevert 
a  clear  shift  towards  the  3f3  type  occurs  (30  percent  com- 
pared to  only  ten  percent  of  the  2f2  type  within  this  cell). 
Seniors  writing  on  this  topic  co^nplete  the  pattern  of  .shift 
towards  the  3f3  typet  where  it  occurs  In  5  out  of  11  papers 
(*6  percentt  along  with  C  strategy  in  20  percent*  and  BC 
strategy  In  20  percent  of  the  remaining  papers  In  the  cell). 
One  hypothesized  explanation  for  the  stronger  trend  towards 
3t3  task  constructions  In  1975  as  compared  to  197<if  is  that 
the  top  ic  con  str  al n  t  of  wr It  I ng  about  a  person  »  rather  than 
a  non-human  "thingt*  favored  the  use  of  relatively  abstract 
language  (since  a  person  necessar  i 1 y  has  thought  s  and  f ee 1  - 
Ings  which  must  be  described  somewhat  abstractly).  Put 
another  way*  it  may  be  that  the  requirement  of  writing  about 
a  *thing*  favors  the  use  of  relatively  ccncretet  descriptive 
1  anguage  t  and  relatively  more  errphasls  on  the  ••thing"  Itself 
than  on  one's   reasons  for  wanting  to  be  "it.* 
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Inc^easeS'frr.'^rpercent  a.ong  fresh.en  to  .P-^J^^JT^^^ 
seniors.      Interest  I  njTy.     too,     the     range  ^ ' ^^"^ 

choien    by    -r  Iters     decreased    notably    frcy.    ^''^hmen  to 
Seniors.      Where    freshmen  responses  varfed  among  5  alterna 
twis  to  the  3.3  preferred  type  (.naking    up     10    Percent  of 
each    cell     total),   sophomore  and  junior  responses  «ere  Uf^ 
Ited  to  A  alternatives,  and  *"J<'r  responses     Included  only 
two    alternatives     (one    story    and  one  II/BC).     It  is 
Jo?ing  here  that  those  writers  who  did  choose  some  comb'na- 
tioi  ^f    2    function    with  a  2  or  3/trategy.  d.d  so  at^he 
Dotential  cost  of  Ignoring  a  strong  topic  constra  nt.  .«f>«se 
writers  focuted.almost  exclusively  on  the  "J^r^"^ 

Ug!  detailing  Its  history)   In  violation  of  the  constraint. 

•Make  it  clear  ttby  humanity  would  be  better  off  without  this 

invention.* 

Again,  we  expected  to  find  a  preference  ^.3  in  this 

topic/year       since     the    given    task    calls  on  the  writer  to 

i^comm^nd-  an  object  and  support  that  / ""JlJt"^? Ji^^^year ! 
♦  ^  ar^    stronalY     favored   in  this  topic/ycart 

tive  types  as  opposed  to  4  alternatives  percent) 
^how  a  much  weaker  preference  for  3.3  In  1977  \^.t  percc  tj 
and  also  show  a  weak  preference  for,2.9's  (27  percent,  dl  - 
cussed  above  as  the  -thought  mirror-  type).  Seniors  again 
show  the  strongest  preference  for  3.3  of  any  of  the  grade 
levels  In  response  to  this  topic 

One  aspect  of  the  different  pattern  of  response  to  1977 
as    Z%rir\o  1976  topic/years.  ?s  that  -"V       '  ,Tnd 

rhe^r%:rpSn%e%-^;^crud:-::iera?-K;;;  s^^^ 

discussed  -how  they  came  to  make  a  e^^.^^^i^ttSn 
object.-  Thus,  there  were  somewhat  ^T^^e  stories  written 
overall  for  this  topic  as  compareds  to  1976  percent  vs. 

?  pe  cent)  and  a  greater  range  of  variation  ^'"eluding  9.9 
types,  and  various  combinations  of  narrative  strategy  -'th 
II   (2)  or  111    (3)  functions). 
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Thfs  topic  gencr^^lly  seemed  to  the  most  complex  in 
terms  of  given  task  constraints.  We  therefore  hypothesized 
that  it  would  elicit  the  widest  range  of  task  constructions 
of  any  of  the  topics;  e.g.  1)  if  the  writer  responded  by 
focusing  on  "the  influence"  (effect  on  life,  changes  in 
life).  the  test  would  probably  have  a  III  function.  2)  if 
the  writer  focused  on  the  Mtnl  of  influence,  he  or  she 
might  produce  a  text  with  function  II.  although  3)  if  the 
aacat  discussed  is  an  t^tatt  a  text  with  function  I  might  be 
produced,  and  4)  if  the  writer  focuses  on  both  the  agent  of 
influence  (an  event)  and  on  the  influence  itself  (e.g. 
chance  In  internal  state  of  the  author,  change  in  self- 
awareness),  the  text  produced  would  be  an  "autobiographical 
essay."  having  a  I-III  function.  In  fact,  responses  to 
topic/year  1978  do  show  the  widest  range  of  variation  of  any 
topic/year.  The  only  strong  preference  shown  ins  for  3.3 
among  sophomores  (50  percent).  Freshmen  show  a  weak  prefer- 
ence for  3.3  as  well  (30  percent);  juniors  show  no  clear 
preference  at  all  (though  3.3  and  4.4 — the  autobiographical 
essay — are  both  represented  at  20  percent).  Seniors  show 
weak  preferences  for  both  3.4  and  9.9  (i.e.  totally  unclas- 
sifiable  papers)  at  27  percent  each.  Overall,  stories  are 
represented  by  15  percent  of  the  papers,  autobiographical 
essays  by  10  percent  (greater  than  for  any  other 
topic/year),  entity  descriptions  (2.2)  by  only  7  percent, 
and  commentary/evaluations/ideas  (3.3)  by  29  percent. 

Finally,  to  suntfrartze  the  overall  results.  the  pre- 
ferred task  construction  by  topic/year  is  as  follows!  1.1 
for  1973  (43  percent);  2.2  for  1974  (26  percent);  3.3  for 
1975  (27  percent);  3.3  for  1976  (60  percent);  3.3  for  1977 
(47  percent);  3.3  for  1978  (29  percent).  As  we  expected, 
preferences  In  each  case  reflect  "primary  types*"  or  conven- 
tionalized associations  of  function  and  strategy.*  Task  con- 
struction correlate  with  grade,  overall,  is  heavily  influ- 
enced by  the  preference  for  3f3  among  all  grades  in  1976.  so 
that  3.3  appears  as  the  preferred  type  for  all  grades,  at 
around  33  percent  of  each.  However,  type  3.3  does  climb  to 
40  percent  of  the  total  senior  papers,  suggesting  that  our 
prediction  of  devel  op^ient  from  I  to  III  with  age  may  have 
some  va 1  id  I ty . 

The  1.1  type,  which  we  expected  to  be  most  preferred  by 
freshmen,  remains  a  fairly  constant  13  -  18  percent  of  the 
total  within  each  grade.  This  seems  due.  in  part,  to  the 
strong  preference  for  the  1.1  type  at  all  grade  levels  in 
response  to  the  1973  topic.  It  is  only  in ^response  to  the 
1975  topic  that  we  can  see  a  clear  possibility  of  grade 
effect  on  task  cons tr jct iont  which  will  have  to  be  taken 
into  account  in  future  case  study  analysis. 

For  the  present,  we  can  say  that  it  Is  possible  to 
Identify    more  or  less  strongly  preferred  task  constructions 

ERIC  '-'J/. 


*or  each  topic/year*  and  to  hypothesize  about  the  interpre- 
tation of  given  task  constraints  which  gave  rise  to  those 
preferred  types.  The  next  step  in  our  analysis  will  be  to 
determine  whether  a  significant  correlation  exists  between 
holistic  score  and  the  use  of  the  preferred  task  construc- 
tion in  a  given  topic/year,  within  a  given  grade  ievcl.  In 
this  way  we  "iope  to  account  for  some  of  the  variance  in  new 
score  not  adequately  explained  by  reference  to  word  count. 
Furthermor et  we  hope  to  be  able  to  demonstratet  through  case 
study  analyslSf  that  what  a  student  actually  dflfiA  i" 
response  to  a  given  topic  will  Influence  the  quality  rating 
of  his  work— not  because  some  task  constructions  are 
inherently  "better*  or  more  "successful"  or  more  "appropri- 
ate" than  otherst  but  because  the  individual  student  may  be 
nore  successful  at  using  a  "tried  and  true"  task  construc- 
tiont  than  he  is  when  attempting  something  new  and  perhaps 
more  complex  or  challenging. 
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APPEin)lCES 


TABLE  1 

Topics  Used  in  Six  Years  of  Drake  Writing  Assessment 


1973       Write  about        event  you  wish  you  had  witnessed  or  could  witness.    The  event 
can  be  real  or  Imagined;  the  time  of  the  event  can  be  past,  present  or 
future      Make  It  clear  why  the  event  Is  significant  to  you.    You  may  write 
a  journal  entry,  letter,  dialogue,  monologue , -essay ,  story,  autobiography, 
or  other  form. 

197^        If  you  had  to  choose  to  be  aomething  other  than  a  human  being,  what  pla^t  or 

anlLl  or  other  form  would  you  choose?    In  your  writing,  give  your  reader  sere 
idea  of  what  you  think  it  would  be  like  to  be  that  form,  and  of  why  y°u 
it      You  may  do  this  vrlting  as  a  journal  entry,  a  letter,  a  dialogue,  a  stcr>  , 
an  autobiography,  an  essay,  a  poem.  etc. 

1973       If  you  could  change  pl.c.s  with  .omeone  who  would  it  be?    The  person  yo-. 

write  .bout  c.n  be  living,  de^d,  dr.wn  from  p.st  or  present,  from  books  films, 
«c      or  frcir  your  own  lMgin«tion.     I"  your  writing  give  your  reader  .one 
Tdea  of  what  it  would  be  like  to  b.  that  person,  and  of  why  that  1  fe  appeals 
to  you.     You  mav  do  this  writing  as  a  journal  entry,  character 
dialogue,  letter,  atory.  autobiographical  essay,  argument,  poer.,  or  other  for-.. 

1976        Not  all  inventions  have  b«en  good  for  all  humanity.  ."^^ ^ 

would  be  better  off  without,  and  make  it  clear  why.     You  ma>  do  this  VTitxr.c 
as  an  essay,  journal,  letter,  story,  or  other  form. 


197 


197S 


Imagine  that  a  small  group  of  people  will  be  aent  to  colonize  a  nev  plare.. 
Food     Clothing,  ahelter  and  transportation  have  been  provided  for.  You 
.re  among  tho';  asked  to  a.l.ct  a  few  additional  things  to  be  sent  along 
'n  thnLited  apace  available  in  the  ahip.     What  one  item  would  you 
iecom^end.  and  why?    You  may  write  your  recommendation  in  the  form  of  a 
story,  a  dialogue,  a  letter,  a  apeech.  an  essay  or  other  form. 

Write  about  aome  way  in  which  your  life  has  beer.,  or  might  be  l^Jl^lT^' 
You  miRht  write  about  the  influence  of  another  person,  a  book  or  f Hit, 
1°  idef  o^an  event  auch  as  a  triumph  or  defeat,  or  a  audden  gain  or  loss. 
^Ik    i  "c  ear  just  what  or  who  influenced  you.  and  what  the  ef  ect  was  upon 

or  what  the  effect  could  be  upon  you.    You  may  do  this  writing  as 
n^u^al  entry,  character  aketch.  dialogue,  letter,  story,  autobiographical 
essay,  or  other  form. 
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IWew  Score !Mctn$  forjOvcrin  Simple  tnd  Indlvlduti  Cohorts 
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TABLE  4 

Topic  Years  according  to  Cohorts 

Grade         9                10  11  12 

Cohort      I                         73             74  75  76 

II                        74            75  76  77 

III                        75            76  77  78 

IV                        78            73  74  75 

V                        77            78  73  74 

VI                        76            77  78  79 
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